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EXECUTIVE SUMMARY 


This report is a descriptive analysis of the Common Core State Standards (CCSS), intended to determine 
which content is eligible for the Consortium's end-of-year summative assessment for English language 
arts (ELA) and mathematics in grades 3-8 and high school. The high school standards analyzed were 
those in grades 9-10 and 11-12 for ELA, and all conceptual categories for mathematics. All high school 
standards were analyzed, since which high school content will be included in the summative grade 11 
assessment has yet to be determined. Outcomes of this analysis are intended as a starting point to 
inform discussion as the Consortium designs its summative assessment and develops its test and item 
specifications. This report is intended to be used with other documents the Consortium has or is 
developing that provide more conceptual, construct-centered, and evidence-based frameworks for its 
assessment design and development. 

Key Questions 

This analysis addressed the following key questions: 

1. Which CCSS are eligible for the SBAC summative assessment? 

2. What is the range of depth of knowledge of the eligible CCSS? 

The primary audience for this report is the content and curriculum specialists and test and item 
development experts who will be involved in the next steps of determining which specific standards 
should be tested on the summative assessment and by which item types, and how the standards should 
be clustered, if at all, for testing multiple standards with given assessment items/tasks, as well as 
developing the test and item specifications and test blueprints. 

Methodology 

In order to determine eligibility of content for summative assessment, content standards were coded 
according to the following criteria, based on the information provided in the CCSS documents: 

1. Learnable within the school year 

2. Expected content for all students at the grade level/span 

Content standards were also coded according to the following criteria, based on expert judgment: 

3. Measurable via on-demand tasks in an end-of-year summative assessment (i.e., item types, 
response types) 

4. Depth of knowledge (DOK) 

A standard would be rated as eligible for the summative assessment if it was rated as "yes" for all of the 
first three criteria (i.e., learnable, expected, and measurable), and would be rated as not eligible if it was 
rated as "no" for any of the first three criteria. 

Findings 

The organization of the standards in the CCSS differs between ELA and mathematics, both in the way the 
content is categorized (e.g., by strand, domain, or conceptual category) and across grade levels/spans 
(ELA has cross-grade College and Career Readiness [CCR] Anchor Standards, whereas the mathematics 
standards are organized based on domains that vary across grade levels/spans according to grade- 
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appropriate content). The results of this study are organized and presented in a manner consistent with 
the organization of each content area in the CCSS. 

ELA 

Of the 333 grade-level standards in grades 3-8 and high school, all standards were judged to be 
learnable during the school year and expected of all students; 285 standards were judged to be 
measurable via on-demand summative assessment and 48 not measurable. The standards judged to be 
measurable via on-demand summative assessment are all eligible; the remaining 48 standards are not 
eligible. 

Eligibility is distributed across the grades as follows: 

• In grade 3, 35 of 42 standards are eligible. 

• In grades 4 and 5, 35 of 43 standards are eligible. 

• In grades 6-12, 36 of 41 standards are eligible. 

The following standards are not eligible at all grades: 

• Reading standard 10 (for both Literature and Informational Text) 

• Writing standards 6 and 10 

• Speaking and Listening standard 1 

Additional standards not eligible in grades 3-5 are: 

• Grades 4 and 5 Reading Foundational Skills standard 3 

• Grades 3-5 Reading Foundational Skills standard 4 

• Grades 3-5 Writing standard 7 

Mathematics 

Of the 316 grade-level and conceptual category-level standards in grades 3-8 and high school, 270 were 
judged to be eligible for the summative assessment; 46 were not eligible. The eligibility was distributed 
across the grades as follows: 

• In grades 3-7, all standards were eligible. 

• In grade 8, 27 of 28 standards were eligible. One grade 8 standard in the Geometry domain was 
ineligible; this standard was judged to be not measurable via on-demand assessment. 

• In high school, 111 of 156 standards were eligible and 45 were not eligible. 

• Of the 45 ineligible high school standards, 43 were not expected of all students. The largest 
number of standards not expected of all students was in the Number and Quantity conceptual 
category, but every conceptual category included some standards not expected of all students. 
The remaining two ineligible standards, both in Geometry, were judged to be not measurable 
via on-demand assessment and more appropriate for classroom assessment. 

Next Steps 

The information presented in this report is a starting point for the Consortium's efforts related to test 
and item development. The Consortium should use this study's data and recommendations to inform its 
further explication of the content to be assessed; that is, in terms of what is to be measured and why, 
what is not to be measured and why not, how to best measure the content, and how to ensure 
accessibility to all students of the content measured. 
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There are numerous considerations and trade-offs to be made when designing a summative assessment 
(e.g., content coverage, emphasis, burden). Therefore, coordinated discussion across Consortium groups 
(e.g., Test Design, Item Development, Technology Approach, and Accessibility and Accommodations 
Work Groups and the content specifications committee) must occur in order to verify the need for and 
conditions under which the eligible content identified in this report should ultimately be included or 
excluded on the summative assessment. 

Subsequent discussions should focus on issues such as whether and how assessable content can be 
reasonably clustered to reduce the number of assessed constructs (i.e., which content could and should 
be clustered), the prioritization of the standards/clusters to be assessed, and the cognitive demands and 
types of evidence required by the standards/clusters to be assessed. These discussions should occur 
within the context of understanding the explicit purpose of the summative assessment, relevant 
cognitive models for the assessed domains (i.e., ELA, mathematics), and what the Consortium ultimately 
hopes to accomplish with this assessment in terms of student learning and achievement. Doing so will 
help to inform the subsequent development of an assessment framework and theory of action that can 
guide the development of a valid summative assessment. 

While considering what students are to learn and the claims to be made about students from the results 
of the summative assessment, it is essential that the characteristics of the students be understood and 
considered— that is, the characteristics of all students who will be administered the summative 
assessment, including English learner (EL) students and students with disabilities (SWDs). These 
students' capacities as well as their challenges must be considered upfront and throughout the test and 
item design and development processes. EL students and SWDs have characteristics (e.g., sensory, 
cognitive, physical, linguistic, socio-cultural) that require certain supports (via accessibility strategies 
and/or accommodations) that interact with assessment content, item formats, and administration 
conditions. Therefore, in order to best ensure the development of a summative assessment that yields 
valid interpretations and appropriate consequences for students, the Consortium's summative design 
and development discussions must always consider all our students. 

This final report incorporates comments received from SBAC member states, work groups, and other 
key constituencies (e.g., Technical Advisory Committee). Suggestions intended to generate and/or guide 
further discussion of this study's data among Consortium work groups and committees (e.g., Test 
Design, Item Development, Technology Approach, and Accessibility and Accommodations Work Groups, 
content specifications committee) are provided in this report. Examples that help to illustrate how to 
navigate and use the information from this analysis (e.g., consideration of the content standards, 
comment codes, DOK) also are provided. 
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SMARTER Balanced Assessment Consortium Common Core State Standards Analysis: 

Eligible Content for the Summative Assessment 

INTRODUCTION 

The SMARTER Balanced Assessment Consortium (SBAC) contracted with the Assessment and Standards 
Development Services (ASDS) program at WestEd to conduct an analysis of the Common Core State 
Standards (CCSS) in English language arts (ELA) and mathematics as part of its broader efforts to define 
the eligible content of the CCSS for its various assessments. This report presents the findings of the 
analysis of eligible content for the summative assessment. 

The report is organized as follows: 

• Introduction, including the purpose of the study, organization of the CCSS, and background on 
the SBAC summative assessment; 

• Methodology, including the analysis criteria and study protocol; 

• Summary of findings for ELA and mathematics; 

• Considerations for next steps, including considerations for content clustering, test design, item 
development, test administration, and use of technology; 

• References; and 

• Appendices, containing standard-level ratings and supplementary information. 

Project Purpose 

This report is a descriptive analysis of the CCSS, intended to determine which content is eligible for the 
Consortium's end-of-year summative assessment for ELA and mathematics in grades 3-8 and high 
school. The high school standards analyzed were those in grades 9-10 and 11-12 for ELA, and all 
conceptual categories for mathematics. All high school standards were analyzed, since which high school 
content will be included in the summative grade 11 assessment has yet to be determined (e.g., by the 
Consortium's Test Design Work Group and content specifications committee). Outcomes of this analysis 
are intended as a starting point to inform discussion as the Consortium designs its summative 
assessment and develops its test and item specifications. This report is intended to be used with other 
documents the Consortium has or is developing that provide more conceptual, construct-centered, and 
evidence-based frameworks for its assessment design and development. 

While all CCSS content is assessable and should be assessed in the classroom throughout the school 
year, important choices need to be made about which standards, or parts of standards, are best 
assessed via the required summative accountability assessment proposed by the Consortium. The 
purpose of this study is to determine the total set of standards eligible for the summative assessment 
only. Given the purpose of this study, and given that its findings are being presented in the early stages 
of the Consortium's assessment design and development activities, the information in this report should 
in no way be interpreted to constrain the Consortium's discussions and decisions. Rather, the findings 
presented in this report and suggested considerations for next steps should help inform the Consortium 
in its further examination and discussion of the study's descriptive data and their implications for the 
design and development of the summative assessment. 

For the purposes of this study, a standard was considered eligible for the summative assessment if it 
was judged to be learnable during the school year, expected of all students, and measurable via the 
range of on-demand assessment item types included in the SBAC proposal. In addition, analysts 
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determined which item types proposed for the summative assessment seem appropriate for assessing 
the content of each standard or part of a standard. Finally, analysts judged the range of cognitive 
complexity of the skills and knowledge described in each standard. The protocol and criteria used in this 
study are described in greater detail in the Methodology section of this report. 

Key Questions 

This analysis addressed the following key questions: 

5. Which CCSS are eligible for the SBAC summative assessment? 

6. What is the range of depth of knowledge of the eligible CCSS? 

The primary audience for this report is the content and curriculum specialists and test and item 
development experts who will be involved in the next steps of determining which specific standards 
should be tested on the summative assessment and by which item types, and how the standards should 
be clustered, if at all, for testing multiple standards with given assessment items/tasks, as well as 
developing the test and item specifications and test blueprints. In particular, this report should be useful 
to the Consortium's Test Design and Item Development Work Groups, along with the content 
specifications committee. 

It is important to underscore that the goal of this analysis was to provide the Consortium's item and test 
developers with information to consider about the assessable content of the CCSS vis-a-vis the 
Consortium's summative assessment. It was not an analysis of the quality of the standards, an unpacking 
of the standards, or an alignment study. The findings of this study are not intended to serve as the 
Consortium's summative test specifications or as a test blueprint. Rather, the findings of this study are 
intended (along with other documents) to provide information that will inform the development of test 
and item specifications and should be considered as providing neither a final or restrictive 
determination nor a compulsory list of individual standards that must be included on the summative 
assessment. The Consortium's work groups, state members, staff, and consultants necessarily engaging 
in further discussion to develop test and item specifications (e.g., the Test Design and Item Development 
Work Groups) should consider the findings of this study as they work towards determining which 
specific content is to be tested and how it should be tested. 

A Note Regarding a Preliminary Review and Comment on the Report 

A preliminary draft version of this report, including all findings, was distributed to key Consortium 
stakeholders for review and comment, in order to best ensure that the information presented in the 
report was clear and the findings usable vis-a-vis the intent and purpose of this analysis and this report. 
These stakeholders included all Consortium State members, Consortium work groups, Executive 
Committee members, and the Consortium's Technical Advisory Committee. All comments received were 
considered and addressed in this final version of the report. 

Many comments required responses that fell beyond the scope of this report. Nonetheless, these 
comments offer direction for possible next steps and would be best addressed with further discussion 
by Consortium work groups and committees, the Executive Committee, and the Technical Advisory 
Committee. Appendix F presents a listing of all comments received from the preliminary review with 
recommendations for follow-up with relevant stakeholder groups. Cross-work group/committee 
communication and coordination are encouraged and noted accordingly in the appendix. 


UUestEd J 


2 


March 4, 2011 


Background and Organization of the Common Core State Standards for English Language Arts and 
Mathematics 

The CCSS were developed through an effort coordinated by the Council of Chief State School Officers 
and the National Governors Association, with the intention of producing a set of common standards that 
are "(1) research and evidence based, (2) aligned with college and work expectations, (3) rigorous, and 
(4) internationally benchmarked" (CCSSO and NGA, 2010a, p. 3). The standards for ELA and mathematics 
have different organizational structures. The ELA standards are organized based on cross-grade College 
and Career Readiness (CCR) Anchor Standards, whereas the mathematics standards are organized based 
on domains that vary from grade to grade according to grade-appropriate content. This subsection 
provides an overview of the organization of the CCSS in each content area in order to provide some 
context for the analysis. 

English Language Arts 

The ELA standards are divided into four strands: Reading, Writing, Speaking and Listening, and 
Language. As explained in the introduction of the CCSS for English Language Arts, "each strand is headed 
by a strand-specific set of College and Career Readiness (CCR) Anchor Standards that is identical across 
all grades" (CCSSO and NGA, 2010a, p. 8). Grade-specific standards corresponding to the CCR Anchor 
Standards are provided for each individual grade from grades 3-8, and for grade spans 9-10 and 11-12 
in high school. Table 1 provides an overview of the organization of the ELA standards by CCR Anchor 
Standard, strand, and grade level. 


Table 1. Organization of the Common Core State Standards for English Language Arts 


K-12 

Anchor Standards 

K-5 

(K, 1, 2, 3, 4, 5) 
Grade-Specific Standards 

6-12 

(6, 7, 8, 9-10, 11-12) 
Grade-Specific Standards 

College and Career Readiness 
Anchor Standards for Reading 

1-10 

Reading Standards 
for Literature K-5 

1-10 

(8 not applicable) 

Reading Standards 
for Informational 
Text K-5 

1-10 

Reading Standards 
for Literature 6-12 

1-10 

(8 not applicable) 

Reading Standards 
for Informational 
Text 6-12 

1-10 


Reading Standards: Foundational Skills K-5 

1-4 (grades K-l) 

3-4 (grades 2-5) 


College and Career Readiness 
Anchor Standards for Writing 

1-10 

Writing Standards K-5 

1-8 (grades K-2) 

1-8, 10 (grade 3) 

1-10 (grades 4-12) 

Writing Standards 6-12 

1-10 

College and Career Readiness 
Anchor Standards for 
Speaking and Listening 

1-6 

Speaking and Listening Standards K-5 

1-6 

Speaking and Listening Standards 6-12 

1-6 

College and Career Readiness 
Anchor Standards for 
Language 

1-6 

Language Standards K-5 

1-6 

Language Standards 6-12 

1-6 


Numbers in the table that are bold-faced are grades/grade spans. Numbers that are italicized are the number of 
the standards. 
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Mathematics 

The CCSS for Mathematics consist of two parts: the Standards for Mathematical Practice and the 
Standards for Mathematical Content. The eight Standards for Mathematical Practice are the same across 
all grades, while the Standards for Mathematical Content vary from grade to grade. For the purposes of 
this study, only the Standards for Mathematical Content were individually analyzed, and from here on, 
"mathematics standards" refers to the Standards for Mathematical Content. 

The Standards for Mathematical Practice describe "processes and proficiencies" that all students should 
develop as they become mathematically proficient (CCSSO and NGA, 2010b, p. 6). These standards are 
intended to be connected to mathematical content in mathematics instruction, particularly as students 
engage with content standards that set an expectation of understanding (p. 8). With respect to 
assessment, key features of the Mathematical Practice standards must be purposefully integrated in 
classroom assessment and in development of SBAC interim/benchmark, formative, and summative 
assessment tasks. Due to the integrative and process nature of these standards, an individual item or 
task would not address a separate Mathematical Practice standard on its own; rather, a collection of 
formative assessment experiences or summative items or tasks would focus on certain aspects of the 
Mathematical Practice standards as applied to concepts of mathematical content to ensure that 
students have opportunities to become mathematically proficient and demonstrate their expertise. 

The mathematics standards are organized by grade level in grades K-8 and by conceptual category in 
high school. In grades K-8, the standards are organized into elementary and middle school (K— 5 and 
6-8, respectively). Each grade level is subdivided into domains, as shown in Tables 2 and 3. 


Table 2. Organization of the Common Core State Standards for Mathematics: 

Elementary School (K— 5) 


K-5 Domains 

Grade Levels with Standards 
in the Domain 

Domain 

Code 

Counting and Cardinality 

K 

CC 

Operations and Algebraic Thinking 

K-5 

OA 

Number and Operations in Base Ten 

K-5 

NBT 

Number and Operations— Fractions 

3-5 

NF 

Measurement and Data 

K-5 

MD 

Geometry 

K-5 

G 


Table 3. Organization of the Common Core State Standards for Mathematics: 

Middle School (6-8) 


6-8 Domains 

Grade Levels with Standards 
in the Domain 

Domain 

Code 

Ratios and Proportional Relationships 

6-7 

RP 

The Number System 

6-8 

NS 

Expressions and Equations 

6-8 

EE 

Functions 

8 

F 

Geometry 

6-8 

G 

Statistics and Probability 

6-8 

SP 
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The high school CCSS for mathematics are organized by conceptual category. Within each conceptual 
category are domains specific to that category, as shown in Table 4. 


Table 4. Organization of the Common Core State Standards for Mathematics: 

High School (9-12) 


9-12 Conceptual Categories /Domains 

Domain 

Code 

Number and Quantity 

The Real Number System 

N-RN 

Quantities 

N-Q 

The Complex Number System 

N-CN 

Vector and Matrix Quantities 

N-VM 

Algebra 

Seeing Structure in Expressions 

A-SSE 

Arithmetic with Polynomials and Rational Expressions 

A-APR 

Creating Equations 

A-CED 

Reasoning with Equations and Inequalities 

A-REI 

Functions 

Interpreting Functions 

F-IF 

Building Functions 

F-BF 

Linear, Quadratic, and Exponential Models 

F-LE 

Trigonometric Functions 

F-TF 

Modeling (Content is integrated with standards in other conceptual categories.) 

Geometry 

Congruence 

G-CO 

Similarity, Right Triangles, and Trigonometry 

G-SRT 

Circles 

G-C 

Expressing Geometric Properties and Equations 

G-GPE 

Geometric Measurement and Dimension 

G-GMD 

Modeling with Geometry 

G-MG 

Statistics and Probability 

Interpreting Categorical and Quantitative Data 

S-ID 

Making Inferences and Justifying Conclusions 

S-IC 

Conditional Probabilities and the Rules of Probability 

S-CP 

Using Probability to Make Decisions 

S-MD 


Background on the SBAC Summative Assessment 

The results of this analysis are intended to inform the design and development of the Consortium's 
summative assessment of the CCSS. The summative assessment is a component of the Consortium's 
balanced assessment system, which includes interim assessment, formative assessment tools and 
practices, and summative assessment. The description of the intent of the Consortium's summative 
assessment, as presented in the SBAC proposal dated June 23, 2010 (SBAC, 2010b), was used to inform 
the parameters established for this analysis. A summary of this description follows. 1 


1 It is expected that the descriptions presented in the SBAC proposal (SBAC, 2010b) will be further elaborated and 
refined by the Consortium (e.g., regarding test format, administration conditions, item types). 
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Purpose of the Summative Assessment 

The summative assessment is intended to measure the "full range of student abilities on the CCSS" in 
ELA and mathematics, providing information on student achievement and growth for accountability 
(SBAC, 2010b, p. 37). As described in the proposal, the SBAC's "system of summative, l/B, and formative 
assessments will produce instructional^ useful information available throughout the instructional year 
to help guide and support differentiated instruction" (p. 41). 

Population 

The summative assessment is intended for all students, including students from special populations 2 . 
"The SBAC system will ensure meaningful accessibility to students; that is, valid forms of presentation, 
engagement, and response for students— to address access barriers including vision, hearing, motor, 
and other sensory, physical, cognitive, processing, and language needs of students— that enable 
students to fully demonstrate what they know and can do" (pp. 47-48). 

Testing Window 

The summative assessment will be administered within 12 weeks of the end of the instructional year. 
Students will be allowed two testing opportunities during the window. 

Grades/Subjects Tested 

The summative assessment will test the Common Core State Standards in English language arts and 
mathematics in grades 3-8 and 11. 3 

Test Format 

The selected-response and constructed-response sections of the summative assessment will be 
administered as a computer-delivered adaptive assessment. "The computer adaptive approach provides 
maximally accurate assessment for each student to better determine whether students are on track for 
being college- and career-ready and to support the measurement of the full range of student abilities" 
(P-42). 

Item Types 

The summative assessment will include the following item types as defined in the SBAC proposal (SBAC, 
2010b): 

• Selected response 

• Extended constructed response 4 

• Technology enhanced 


2 However, the assessment is not intended for students with severe cognitive disabilities. 

3 For this project, eligible content was defined for ELA and mathematics in grades 3-8 and high school. The high 
school standards analyzed were those in grades 9-10 and 11-12 for ELA, and all conceptual categories for 
mathematics. 

4 "Constructed response" is a general term for items requiring the student to generate a response as opposed to 
selecting a response. As defined in the SBAC proposal, "extended constructed response" describes assessment 
tasks such as writing a first draft in ELA or solving a multi-step math problem that has more than one possible 
answer or more than one solution method. The general term "constructed response" includes "extended 
constructed response" items as well as items requiring response by short answer, grid-in, and short answer with an 
explanation. 
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Performance task (two at each of grades 3-8, up to six by grade 11) 


For the purposes of this study, two further item type descriptors were identified for ELA only: 

• Writing prompt (an item type designed to measure writing and language standards) 

• Oral response required (a response type that applies to technology-enhanced items to measure 
speaking skills) 

The numbers and types of items will vary across grades and content areas. Descriptions of the item 
types from the SBAC proposal and other SBAC supporting documents are included in Appendix B. 
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METHODOLOGY 


This section describes the methodology used in this study and begins with a discussion of the guiding 
principles and considerations that framed this study's analyses. Next, the study criteria, coding protocol, 
and training and calibration process are described, followed by decision rules and descriptions of the 
analysts' qualifications. 

Guiding Principles and Considerations 

Prior to analyzing the Common Core State Standards (CCSS), guiding principles and considerations were 
established to help define and make as explicit as possible the parameters of this study, to determine 
which criteria would be most appropriate for this study's analyses, and to inform the reporting of the 
results of the analyses. The guiding principles and considerations are presented in Appendix A. 

Study Criteria 

Based on the guiding principles and considerations, as well as the study's goals, WestEd developed an 
analysis protocol to identify content in the CCSS that could be considered eligible for summative 
assessment. This section describes that protocol and the criteria used in this study by the analysts. 

This analysis focused on determining eligible content for summative assessment. Content standards 
were coded according to the following criteria, based on the information provided in the CCSS 
documents: 

7. Learnable within the school year 

8. Expected content for all students at the grade level/span 

Content standards were also coded according to the following criteria, based on expert judgment: 

9. Measurable via on-demand tasks in an end-of-year summative assessment (i.e., item types, 
response types) 

10. Depth of knowledge (DOK) 

A standard would be rated as eligible for the summative assessment if it was rated as "yes" for all of the 
first three criteria (i.e., learnable, expected, and measurable), and would be rated as not eligible if it was 
rated as "no" for any of the first three criteria. 

Each standard was analyzed according to the criteria described in this section. This level of analysis was 
selected because it was conducive as a starting point towards yielding information that could inform 
item specifications and, for example, aggregation or clustering of skills. The standard-by-standard 
analysis does not suggest that the summative assessment be composed of one item per assessable 
standard, nor does this analysis suggest the assessment test discrete skills. As explained previously in 
this report, the intention of the information yielded by this protocol and its criteria is to inform 
discussions about the Consortium's test and item design and development. Further discussion by the 
Consortium about the content and design of the summative assessment is expected. 
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The four criteria were operationalized as follows. 


1. Learnable Within the School Year 

Unless otherwise stated in the CCSS documents, content was considered learnable within the school 
year 5 ; that is, a school year (or a course within a school year) would provide adequate time for a student 
to learn the content. 6 Analysts thoroughly reviewed the CCSS for each grade level/span. For each 
content standard, analysts indicated whether: 

• content is learnable within the school year (Yes [Y]); or 

• CCSS documents indicate that content is not intended to be learned within the year (No [N]). 7 

If the standard was coded N, analysts recorded the specific CCSS source that defines the content 
standard as not intended to be learned within the school year. 

2. Expected Content for All Students at the Grade Level/Span 

Unless otherwise stated in the CCSS documents, content was considered expected for all students at the 
grade level/span; that is, the content is part of what all students should know or be able to do by the 
end of the given grade level or grade span. Analysts thoroughly reviewed the CCSS for each grade 
level/span. For each content standard, analysts indicated whether content was: 

• expected for all students at the grade level/span (Yes [Y]); or 

• not expected for all students at the grade level/span, as indicated in the CCSS documents (No 
[N]). 

If the standard was coded N, analysts recorded the specific CCSS source that defines the content 
standard as not expected for all students at the grade level/span. 

3. Measurable Via On-Demand Tasks in an End-of-Year Summative Assessment 

A standard was considered measurable via on-demand summative assessment tasks if it can be assessed 
by any of the item types listed in the following subsection, as defined in the SBAC proposal (SBAC, 

2010b, pp. 42, 52-53). An on-demand task is one that is presented to the student apart from 
instructional activities and requires the student to provide a specified type of response within the 
defined time and place parameters of the assessment tool. A standard was considered to not be 
measurable via on-demand tasks if assessment of the standard requires extended time to complete and 
cumulative collection of results (e.g., portfolio; research project over weeks or months), or if it is best 
assessed via in-person observation or interaction (e.g., the content needs to be embedded in classroom 
discussion; the content targets interactive skills). Analysts reviewed each content standard and indicated 
whether and how each standard was most efficiently measurable using the following criteria. 


5 This determination focuses on whether each standard analyzed is teachable within a year and not whether the 
collection of standards in the CCSS is teachable within a year. 

6 Additional analysis by content and curriculum experts will need to be conducted to determine the specific 
content expected to be learned by the time of the testing window for the end-of-year assessment. 

7 Note that this criterion results in the mathematics standards marked "+" not being considered eligible for the 
summative assessment, since these standards address advanced content that is not expected of all students to 
study in order to be college and career ready (CCSSO and NGA, 2010b, p. 57). Exclusion from the summative 
assessment precludes these standards from being assessed by the Consortium's other assessments (e.g., 
formative). 
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Item Type 


The SBAC summative assessment will be designed to include problem-solving and performance tasks 
with some extended responses, to be completed over a number of class periods. Therefore, for each 
standard, analysts indicated the type(s) of items from the following list that could be used to assess the 
standard on a summative assessment. That is, for each item type, analysts were asked whether, in their 
professional judgment, an item of that type could generate sufficient evidence to assess the standard or 
a key component of the standard. Descriptions of the item types from the SBAC proposal and other 
SBAC supporting documents are included in Appendix B. 8 Based on these descriptions, when coding 
standards as measurable by technology-enhanced items or performance tasks, analysts considered it 
likely that a set of multiple standards would be combined when assessed by these more complex item 
types. Analysts selected from the following item types: 

ELA— Reading Item Types 

• Selected response (multiple choice) 

• Extended constructed response 

• Technology enhanced (multiple choice or constructed response) 

• Performance task 

ELA— Writing, Speaking and Listening, and Language Item Types 

Writing/Language 

• Selected response (multiple choice) 

• Technology enhanced (multiple choice or constructed response) 

• Writing prompt 

• Performance task 

Speaking and Listening 

• Technology enhanced (multiple choice or constructed response) 

• Oral response required (an item assessing the standard would require an oral response in 
order to address the content of the standard) 

Mathematics Item Types 

• Selected response (multiple choice) 

• Extended constructed response 

• Technology enhanced (multiple choice or constructed response) 

• Performance task 

4. Depth of Knowledge (DOK) 

Analysts reviewed each standard to determine the range of cognitive complexity required to perform 
the skill or demonstrate the knowledge described by the standard. Analysis included consideration of 
the descriptors in the standard as well as the ways in which students would show what they know and 
can do with respect to the skills and knowledge described. Norman Webb's (2005) four levels of 
cognitive complexity were used for this analysis. These levels were developed specifically for K-12 
standards and alignment studies and are widely used in alignment studies throughout the nation. In 


Because the item type definitions require further elaboration and refinement (e.g., by the Item Development, 
Technology Approach, and Accessibility and Accommodations work groups), analysts were not able at the time of 
this analysis to prioritize among or recommend the "most appropriate" item type to a assess a given standard. 
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addition, the descriptors for each level include specific examples for ELA and mathematics content, 
aiding in consistency of ratings. 

The four levels are listed below; see Appendix C for the complete descriptions of the levels as they apply 
to ELA and mathematics. 

• Level 1 (Recall) 

• Level 2 (Basic Application) 

• Level 3 (Strategic Thinking) 

• Level 4 (Extended Thinking) 

Because some standards describe skills at multiple levels of complexity (e.g., when there are multiple 
skills in a standard that could be applied at different levels of complexity), analysts indicated all 
applicable DOK levels to indicate the full range of DOK. 

The findings from DOK ratings provide information on the range of cognitive complexity of content in 
the standards, which groups engaging in the development of test and item specifications, the test 
blueprint, and item templates can use to inform discussions about which content to test via which item 
types, and whether and how to cluster standards for summative assessment. It is expected that 
Consortium work groups (e.g., Item Development) and committees (e.g., those involved in content 
specifications) will use this information to specify and operationalize the assessed content. 

Eligibility 

Following the coding of each standard for all criteria, analysts made an overall determination of 
eligibility. For a standard to be eligible, all of the following must apply: 

• Learnable within the school year 

• Expected content for all students at the grade level/span 

• Measurable via on-demand tasks in an end-of-year summative assessment 

Comments 

In addition to the codings for each criterion, analysts provided comments when additional information 
about the coding was warranted. Analysts proposed content and wording of comments throughout the 
coding process. These proposed comments were reviewed and refined to be applicable across grades 
and content areas, as appropriate, and the wording of the comments was then standardized to ensure 
consistent use across analysts. A complete list of comments and comment codes is provided in 
Appendix D. 

Coding Protocol 

This study employed a double-rater "read-behind" consensus model, accompanied by ongoing 
calibration between analysts. For each grade or conceptual category, one analyst independently coded 
the standards. A second analyst then reviewed the outcomes of the first analyst's ratings and noted 
agreement or disagreement with the first analyst's ratings. The two analysts then discussed any 
discrepancies between their interpretations as necessary. When discrepancies arose between the 
ratings of analysts with respect to the criteria and coding dimensions (i.e., learnable during the school 
year, expected of all students, measurable via on-demand assessment, eligible for the summative 
assessment, response type, and DOK), analysts discussed their ratings and reached consensus. Analysts 
also reviewed ratings across grade levels and (in mathematics) conceptual categories to ensure that the 


UUestEd J 


li 


March 4, 2011 


protocol was applied consistently. Following coding, project leads reviewed a sample of codes as a 
further confirmation of consistency of coding across grade levels and conceptual categories. The 
complete set of standard-level codes is included in Appendix E. 

This protocol was selected for use given that the specific outcomes of this particular study were not 
intended as the end point for high-stakes use, but instead were intended for descriptive purposes to 
inform further discussions of the CCSS content and its implications for the Consortium's summative 
assessment. This model also was selected to be suitable for the development timeline of the 
Consortium. 

Training and Calibration 

Analysts were thoroughly trained in the study protocol. Analysts from both content areas began training 
together, receiving an introduction to the goals and purpose of the study and an in-depth discussion of 
the study criteria, including the DOK level descriptions. This was done to ensure a common 
understanding of the study criteria and procedures and to best ensure the accuracy and consistency of 
the application of the criteria by analysts in each content area. Mathematics and ELA analysts then 
worked separately to practice applying the protocol to their respective content areas and to calibrate 
their ratings. 

Following the introduction and review of the study criteria, analysts independently coded a set of 
approximately 5-10 pre-selected standards, representative of a range of grade levels and types of 
standards, and discussed their ratings as a group. Then, analysts independently coded approximately 
5-10 additional pre-selected standards and compared their results to those of the content lead. Any 
discrepancies in ratings were discussed, and analysts reached consensus, documenting decision rules as 
necessary. 

Training was considered complete and calibration achieved when accuracy and consistency of 
application of the criteria and implementation of the procedures by all analysts were assured to the 
satisfaction of the content and project leads. 

Decision Rules 

When necessary, analysts developed or refined decision rules to document and standardize their 
interpretations. Decision rules are guidelines related to the application of the criteria and the 
interpretation of a standard, construct, piece of content, or skill (i.e., its operational definition). WestEd 
analysts developed decision rules to ensure the accurate and consistent application of the alignment 
criteria. The decision rules implemented in this study follow. 

General 

1. Verbs in the standards that describe cognitive processes (e.g., "understand," "determine") are 
coded as measurable by the coded item types if the student is able to demonstrate the ability to 
carry out the cognitive process in a way that is measurable according to this study's criteria. For 
example, in mathematics, the student could show understanding of an operation by identifying 
the result of carrying out the operation; in ELA, the student could demonstrate the ability to 
determine the central idea of a text by identifying or explaining the central idea. 

2. Verbs in the standards that describe activities the student engages in are coded as measurable 
by the coded item types if the student is able to carry out the activity in a way that is 
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measurable according to this study's criteria. For example, in mathematics, for the verb "draw [a 
graph]," the student is expected to draw a graph as the assessment task; in ELA, for the verbs 
"trace and evaluate [the argument and specific claims]," the student is expected to trace and 
evaluate the argument and specific claims orally or in writing in a speaking or writing task. 

3. Standards are coded to item types that may measure all or part(s) of the standard. 

ELA Only 

1. For the purposes of an on-demand summative assessment, the term "text" is interpreted to 
refer to an intact piece of authored writing (for example, a complete essay, or an excerpt that 
can stand alone), which may include pieces of writing that are written, edited, or revised for the 
purposes of the assessment. 

2. The item type "on-demand writing prompt" is interpreted to refer to a task requiring the 
student to write for the purpose of assessing the student's skills or knowledge in writing (that is, 
the Writing and Language standards), and not for the purpose of assessing the student's skills or 
knowledge in reading (that is, the Reading standards). 

Mathematics Only 

1. Extended constructed-response item types are interpreted as items that require a student 
response beyond selecting an answer from a provided list (multiple choice). Examples include, 
but are not limited to, gridded-response, short-answer, open-ended, open-response, or 
constructed-response items. 

2. The phrase "verbal statements" in the standards is interpreted as inclusive of written 
statements. 

Project Staff Roles and Responsibilities 

The WestEd staff involved in this study all have expertise and experience in classroom teaching as well 
as standards and assessment development and evaluation, including in-depth understanding of the 
CCSS. The project director, Dr. Edynn Sato, was responsible for working with SBAC leaders to 
conceptualize the study vis-a-vis the Consortium's needs and planned development activities, directing 
and monitoring all aspects of the study, ensuring that the processes of analysis and reporting of the 
findings were grounded in research and best practices, and finalizing the study's report. The project 
leads, Dr. Rachel Lagunoff and Mr. Peter Worth, were responsible for designing the analysis protocol 
and procedures, leading the training of analysts and overseeing their ongoing calibration, interpreting 
the findings, and drafting report sections. The lead content analysts were responsible for rating 
standards and guiding consensus discussions with the other analyst(s), and finalizing decision rules 
agreed on by the analysts. The content analysts were responsible for rating standards, contributing to 
consensus discussion, and determining decision rules with the lead content analysts. Biographies of the 
analysts are provided below. 

Qualifications of Analysts 

The expertise of the analysts in standards analysis and in their respective content areas is essential to 
the quality of this analysis. Analysts in this study collectively have experience and expertise in analysis of 
the CCSS for states (e.g., content crosswalks, alignment), the CCSS content areas (i.e., English language 
arts and mathematics), standards and assessment evaluation and development, curriculum, instruction, 
and assessment alignment and development. Each has also served as a classroom teacher. A brief 
description of the qualifications of each analyst follows. 
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Beverly Nedrow, English Language Arts Lead Analyst 

Beverly Nedrow, Director of English Language Arts Development for the Assessment and Standards 
Development Services (ASDS) program at WestEd, oversees the development of reading and language 
assessments. She works with staff to produce item and passage specifications, training materials, and 
ancillary materials. She also provides content expertise to the ASDS research division for studies related 
to the alignment of state standards to the CCSS. 

Prior to employment at WestEd, Ms. Nedrow served as the Director of English Language Arts 
Development, Senior Director of Educational Product Development, and National Consultant for English 
Language Arts at Harcourt Assessment, Inc. She has been responsible for the development of a 
comprehensive online assessment system that includes both summative and formative assessments in 
reading and mathematics in grades 3 through 8, standardized achievement and diagnostic assessments 
in reading and language, and a number of reading and language assessments for customized state tests 
developed to measure the states' academic standards in compliance with the No Child Left Behind 
legislation. In 1998, she received the President's Circle of Excellence Award for her outstanding 
contribution to Harcourt Assessment, Inc. Ms. Nedrow has also been a teacher and curriculum director, 
and has taught in Robstown and Paris, Texas; Owensboro, Kentucky; and Ras Tanura, Saudi Arabia. 

Ms. Nedrow holds an M.S. in Curriculum and Instruction with Specialization in Reading and English as a 
Second Language from Texas A&M University, and a B.S. in Elementary Education, English and History, 
from Murray State University. 

Elizabeth Jameyson, English Language Arts Analyst 

Elizabeth Jameyson is an Assessment Specialist in Reading in WestEd's ASDS program. Ms. Jameyson has 
written and edited items aligned to individual state standards as well as to the CCSS. Additionally, she 
has worked as an analyst in studies of the alignment of the CCSS to individual state standards, and has 
facilitated item writing and content and bias review meetings. Previously, Ms. Jameyson taught English 
language arts at the middle school and high school levels in Oakland, California. 

Ms. Jameyson holds a B.A. in English literature and sociology from Mills College and an M.A. in teacher 
education and a single-subject English teaching credential from the University of California, Berkeley. 

Scott Firkins, Mathematics Lead Analyst 

Scott Firkins is Associate Director of Mathematics Development in WestEd's ASDS program. He 
supervises editorial and item-writing staff working on item development in WestEd's large-scale 
mathematics assessment projects. Mr. Firkins has served as mathematics content lead on assessment 
projects in Kentucky, Pennsylvania, Utah, and West Virginia. As content lead in Kentucky, he oversaw 
development of items assessing the CCSS for Mathematics in grades 3-8. Mr. Firkins also edits and 
reviews mathematics test items for other assessment projects, including that of Nevada. He constructs 
test forms, develops item specifications, and facilitates content reviews and data reviews with teacher 
committees. He has developed assessment items and facilitated their review by teacher committees for 
alternate assessments based on modified academic achievement standards for the Kansas Assessment 
of Modified Measures (KAMM) and the Pennsylvania System of School Assessment-Modified (PSSA-M), 
and has developed assessment items for the Keystone Exams, Pennsylvania's end-of-course 
assessments. Additionally, Mr. Firkins contributes to alignment studies for standards and assessments. 
He served as lead analyst in a comparison study between the CCSS for Mathematics and the Louisiana 
Big Ideas. Prior to joining WestEd, Mr. Firkins was a middle school and high school math teacher for over 
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nine years, and a curriculum supervisor and director of curriculum and assessment for over three years, 
in Owensboro and Glasgow, Kentucky. 

Mr. Firkins holds a Certification: Rank I in Supervision K-12, an M.A. in Secondary Education, and a B.A. 
in Mathematics from Western Kentucky University. 

Mary Koehler, Mathematics Analyst 

Dr. Mary Koehler has a strong mathematics background combined with classroom experience and 
research experience at both the K-12 and college levels, in addition to a thorough knowledge of K-12 
mathematics curriculum. Dr. Koehler taught in Panorama City, California. She has an understanding of 
the complexities of curriculum and instruction in a variety of settings through experience working with 
teachers and students in rural, suburban, and urban areas. She has experience developing large-scale 
mathematics assessment items and facilitating item content reviews, with particular expertise at the 
high school level. She also has served as a mathematics analyst for alignment studies. 

Dr. Koehler holds a Ph.D. in Curriculum and Instruction— Mathematics Education from the University of 
Wisconsin, Madison; an M.A. in Mathematics with a Specialization in Applied Mathematics (Statistics) 
from the University of California, Los Angeles; and a B.A. in Mathematics from San Diego State 
University. 

Ann Muench, Mathematics Analyst 

Ann Muench is a Senior Research Associate and Mathematics Specialist with WestEd's ASDS program, 
and has worked extensively with state mathematics content standards analyses and alignment studies in 
numerous states, including analyses of the CCSS. Her work includes standards and assessments for both 
general-education and special-populations students. At the national level, she developed mathematics 
standards for Job Corps student training. In the area of assessment development, Ms. Muench works on 
test item development for several state tests by editing and reviewing items for mathematics content 
and bias, and facilitating teacher item writing and review groups. As part of the WestEd Eisenhower 
Regional Consortium, she collaborated with key state, district, and site personnel to provide technical 
assistance and staff development in the areas of mathematics education, assessment, and using data to 
effect change. Ms. Muench has extensive experience in developing educational products; presenting at 
local, state, regional, and national events; and facilitating educational dialogue and discussion using 
tools and processes she co-developed. She is a veteran mathematics teacher, and has taught in Aurora, 
Colorado and Indianapolis, Indiana. She has also written and edited both student and teacher textbook 
materials, as well as assessment tasks in mathematics and career training. 

Ms. Muench completed doctoral studies in Educational Psychology at the University of Colorado, 
Boulder, and holds an M.S. and B.S. in Mathematics from Purdue University. 
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FINDINGS 


The organization of the standards in the CCSS differs between ELA and mathematics, both in the way the 
content is categorized (e.g., by strand, domain, or conceptual category) and across grade levels/spans 
(ELA has cross-grade College and Career Readiness [CCR] Anchor Standards, whereas the mathematics 
standards are organized based on domains that vary across grade levels/spans according to grade- 
appropriate content). Therefore, the results of this study are organized and presented in a manner 
consistent with the organization of each content area in the CCSS. For more information about the 
organization of the CCSS, please refer to the CCSS for English Language Arts and Mathematics (CCSSO 
and NGA, 2010a, 2010b) as well as the Background and Organization of the Common Core State 
Standards for English Language Arts and Mathematics subsection in the Introduction section of this 
report. 

ELA Findings 

This subsection presents a summary of the overall findings for the analysis of eligible content in ELA. 
General findings for the three eligibility criteria are presented first, followed by depth of knowledge 
results. Further details on eligible item types, analyst comments, and range of depth of knowledge are 
presented in the following Summary of Findings by CCR Anchor Standard subsection. 

Discussion of this report's findings is by SBAC work groups and committees (e.g., Item Development; 
Performance Tasks; Test Design; Technology Approach; Accessibility and Accommodations; 
Administration; content specifications) is expected as they undertake further work in designing and 
developing the summative assessment. 

Eligibility 

As described in detail in the Methodology section of this report, standards were considered eligible for 
the summative assessment if they were judged to be learnable during the school year, expected of all 
students, and measurable via on-demand assessment. Table 5 shows the number of standards at each 
grade level/span that were judged to be eligible or not eligible, as determined by the three criteria for 
eligibility. 


Table 5. Number of ELA CCSS Eligible for Summative Assessment 


Grade 

Total 

Learnable 

Expected 

Measurable 

Eligible 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

3 

42 

42 

0 

42 

0 

35 

7 

35 

7 

4 

43 

43 

0 

43 

0 

35 

8 

35 

8 

5 

43 

43 

0 

43 

0 

35 

8 

35 

8 

6 

41 

41 

0 

41 

0 

36 

5 

36 

5 

7 

41 

41 

0 

41 

0 

36 

5 

36 

5 

8 

41 

41 

0 

41 

0 

36 

5 

36 

5 

9-10 

41 

41 

0 

41 

0 

36 

5 

36 

5 

11-12 

41 

41 

0 

41 

0 

36 

5 

36 

5 

TOTAL 

333 

333 

0 

333 

0 

285 

48 

285 

48 

Percent of Total 

100% 

0% 

100% 

0% 

86% 

14% 

86% 

14% 
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Of the 333 grade-level standards in grades 3-8 and high school, all standards were judged to be 
learnable during the school year and expected of all students; 285 standards were judged to be 
measurable via on-demand summative assessment and 48 not measurable. The standards judged to be 
measurable via on-demand summative assessment are all eligible; the remaining 48 standards are not 
eligible. 

Eligibility is distributed across the grades as follows: 

• In grade 3, 35 of 42 standards are eligible. 

• In grades 4 and 5, 35 of 43 standards are eligible. 

• In grades 6-12, 36 of 41 standards are eligible. 

The following standards are not eligible at all grades: 

• Reading standard 10 (for both Literature and Informational Text) 

• Writing standards 6 and 10 

• Speaking and Listening standard 1 

Additional standards not eligible in grades 3-5 are: 

• Grades 4 and 5 Reading Foundational Skills standard 3 

• Grades 3-5 Reading Foundational Skills standard 4 

• Grades 3-5 Writing standard 7 

Further details on eligibility by CCR Anchor Standard are presented in the following Summary of Findings 
by CCR Anchor Standard subsection. 

Measurability by Item Type 

All standards determined to be measurable via on-demand summative assessment were then coded for 
the types that could be most effectively and efficiently 9 used to assess those standards. In ELA, the types 
of items included in the analysis were selected response, extended constructed response, technology 
enhanced, performance task, and writing prompt. In addition, standards were rated for whether an oral 
response would be required for the assessment task. These ratings do not indicate that a single item 
type could necessarily assess the full depth and breadth of the standard, but instead indicate that at 
least a component of the standard is assessable using the item type. Table 6 shows, for the eligible 
standards, the item types by which the standards were judged to be measurable. 


Table 6. Item Types by Which Eligible ELA Standards Were Judged to Be Measurable 


Grade 

Total 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance 

Task 

Writing 

Prompt 

Oral Response 
Required 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

3 

35 

23 

12 

17 

18 

35 

0 

30 

5 

8 

27 

3 

32 

4 

35 

23 

12 

17 

18 

35 

0 

30 

5 

10 

25 

4 

31 

5 

35 

23 

12 

17 

18 

35 

0 

30 

5 

9 

26 

4 

31 

6 

36 

24 

12 

17 

19 

36 

0 

31 

5 

10 

26 

4 

32 


9 The efficiency criterion considered the parameters of the summative assessment as described in the SBAC 
proposal (e.g., range of time typically allotted for a summative assessment; the testing window). 
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Grade 

Total 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance 

Task 

Writing 

Prompt 

Oral Response 
Required 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

7 

36 

25 

11 

17 

19 

36 

0 

31 

5 

10 

26 

4 

32 

8 

36 

25 

11 

17 

19 

36 

0 

31 

5 

10 

26 

4 

32 

9-10 

36 

25 

11 

17 

19 

36 

0 

31 

5 

10 

26 

4 

32 

11-12 

36 

25 

11 

17 

19 

36 

0 

31 

5 

10 

26 

4 

32 

Total 

285 

193 

92 

136 

149 

285 

0 

245 

40 

77 

208 

31 

254 

Percent of Total 

68% 

32% 

48% 

52% 

100% 

0% 

86% 

14% 

27% 

73% 

11% 

89% 


Of the 285 eligible standards, all were judged to be assessable via technology-enhanced items, 245 via 
performance tasks, 193 via selected-response items, 136 via extended constructed-response items, and 
77 via writing prompts. Four Speaking and Listening standards at grades 4-12 and three at grade 3 were 
judged to require an oral response for assessment, since the standards specify that the student speaks 
or presents information and ideas orally. Further details on item types by CCR Anchor Standard are 
presented in the following Summary of Findings by CCR Anchor Standard subsection. 

Depth of Knowledge 

As described in the Methodology section of this report, each standard was analyzed to determine the 
range of depth of knowledge levels in the standard. Table 7 shows, for all standards, the number of 
standards to which each depth of knowledge level was assigned. Table 8 shows, for all eligible 
standards, the number of standards to which each depth of knowledge level was assigned. 


Table 7. Depth of Knowledge of All ELA Standards 


Grade 

Total 

DOK Level 

1 

2 

3 

4 

3 

42 

27 

34 

27 

10 

4 

43 

25 

32 

31 

11 

5 

43 

22 

36 

33 

16 

6 

41 

18 

30 

33 

16 

7 

41 

19 

30 

33 

19 

8 

41 

18 

30 

33 

19 

9-10 

41 

18 

30 

33 

19 

11-12 

41 

18 

30 

33 

22 

TOTAL 

333 

165 

252 

256 

132 

Percentage of Total Standards at DOK Level 

(Standards may cover a range of DOK levels) 

50% 

76% 

77% 

40% 
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Table 8. Depth of Knowledge of Eligible ELA Standards 


Grade 

Total 

DOK Level 

1 

2 

3 

4 

3 

35 

20 

27 

24 

7 

4 

35 

18 

26 

28 

9 

5 

35 

16 

30 

29 

12 

6 

36 

13 

25 

31 

15 

7 

36 

14 

25 

31 

18 

8 

36 

13 

25 

31 

18 

9-10 

36 

13 

25 

31 

18 

11-12 

36 

13 

25 

31 

21 

TOTAL 

285 

120 

208 

236 

118 

Percentage of Total Standards at DOK Level 

(Standards may cover a range of DOK levels) 

42% 

73% 

83% 

41% 


The pattern for DOK levels is similar for all standards and eligible standards. Across all grades, the 
majority of standards were coded to DOK Levels 2 and 3, with the number coded to DOK Level 2 
decreasing slightly and the number coded to DOK 3 increasing slightly from the elementary grades to 
the secondary grades. Standards coded to DOK Level 4 increased from grades 3 through 6, and became 
constant between grades 7 and 9-10, rising slightly at grades 11-12. Standards coded to DOK Level 1 
followed the reverse pattern, decreasing from grades 3 through 5, and remaining about the same at 
grades 6 through 12. 

Depth of knowledge can also be examined by looking at the range of depth of knowledge of the 
standards. Range of depth of knowledge is presented for standards at each grade level by CCR Anchor 
Standard in the following subsection. 

Summary of Findings by CCR Anchor Standard 

The CCSS for English Language Arts are organized across grade levels by the Career and College 
Readiness (CCR) Anchor Standards, which are intended to help show how content and concepts change 
and progress across the grade levels. To support consistent interpretation of the content of the 
standards for assessment purposes, this subsection presents the results of the analysis for eligible 
content by CCR Anchor Standard, grouping the standards into their conceptual subdivisions (for 
example, the first three Reading standards are grouped as addressing "Key Ideas and Details"). 

Overall, as seen in the following tables in this subsection, the results for eligible item types are 
consistent across grade levels. The range of DOK across grade levels is provided for all eligible standards 
or sets of standards. This range may be useful during the development of item specifications and item 
templates, to ensure that the appropriate cognitive depth of content is addressed in the items, based on 
grade level, item type, and selection of standards to be clustered (for items that address multiple 
standards). 

In some cases, the DOK ranges for the same Anchor Standard may vary across grade levels. In such 
cases, special consideration may need to be taken in interpreting the intent of the standards in order to 
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represent progression of skills across grade levels (DOK levels for each grade-level standard were 
determined by analysts based on the particular use of verbs and other relevant descriptors in the 
standard). 

For each of the following sets of CCR Anchor Standards, the text of the standards is provided first, 
followed by tables showing the item types, comment codes, and DOK ranges for the eligible standards. 
Following each table is discussion of inconsistencies across grades, and discussion of the comments. Any 
standards that were determined to be not eligible are followed by an explanation. 

The codes for the ELA standards were determined as follows: “Individual CCR anchor standards can be 
identified by their strand, CCR status, and number (R.CCR.6, for example). Individual grade-specific 
standards can be identified by their strand, grade, and number (or number and letter 10 , where 
applicable), so that Rl.4.3, for example, stands for Reading, Informational Text, grade 4, standard 3 and 
W.5.1a stands for Writing, grade 5, standard la" (CCSSO and NGA, 2010a, p. 8). References to grade- 
specific standards that apply across grades are indicated by replacing the grade position with an "x," so 
that RL.x.2, for example, stands for Reading, Literature, unspecified grade, standard 2. 

This section also includes the following acronyms for item types: 

• SR— Selected response 

• ECR— Extended constructed response 

• TE— Technology enhanced 

• PT— Performance task 

• WP— Writing prompt 

• OR— Oral response required 

Comment codes that appear in the tables are defined in the text following each set of tables, as well as 
in Appendix D. 

Reading Standards for Literature and Informational Text, Grades 3-12 

Key Ideas and Details 

1. Read closely to determine what the text says explicitly and to make logical inferences from it; 
cite specific textual evidence when writing or speaking to support conclusions drawn from the 
text. 

2. Determine central ideas or themes of a text and analyze their development; summarize the key 
supporting details and ideas. 

3. Analyze how and why individuals, events, and ideas develop and interact over the course of a 
text. 


10 For the purposes of this study, lettered content was not coded, although the content was considered, as it 
informed understanding of the numbered standard. 
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Table 9. Summary of Reading Standards for Literature Coding (Part 1) 


Literature 

Standard 

Eligible Item Types 

Comments* 

DOK Range** 

RL.x.l 

SR, ECR, TE, PT 

301 (grades 3, 5, 7) 
308 

1-3 

RL.x.2 

SR, ECR, TE, PT 

301 

2-3 (grades 3-10) 



308 

2-4 (grades 11-12) 

RL.x.3 

SR, ECR, TE, PT 

308 (grades 3-6) 

1- 3 (grades 3, 4) 

2- 3 (grades 5-10) 
2-4 (grades 11-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


Table 10. Summary of Reading Standards for Informational Text Coding (Part 1) 


Informational 

Text 

Standard 

Eligible Item Types 

Comments* 

DOK Range** 

Rl.x.l 

SR, ECR, TE, PT 

301 (grade 3) 
308 

1-3 

Rl.x.2 

SR, ECR, TE, PT 

301 

308 

1- 3 (grade 3) 

2- 3 (grades 4-6, 9-10) 
2-4 (grades 7, 8, 11-12) 

Rl.x.3 

SR, ECR, TE, PT 

308 (grades 3, 6) 

2-3 (grades 3, 7-12) 

1- 3 (grades 4, 5) 

2- 4 (grade 6) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


For both literature and informational text standards, all four item types proposed for the reading 
assessment are eligible item types for all standards at all grades, though analysts noted that, for many of 
the grade-level standards, "at least one verb in the standard requires the student to generate a 
response" (comment code 301) and/or "full coverage of the standard may not be possible with selected 
response" (comment code 308). 

Craft and Structure 

4. Interpret words and phrases as they are used in a text, including determining technical, 
connotative, and figurative meanings, and analyze how specific word choices shape meaning or 
tone. 

5. Analyze the structure of texts, including how specific sentences, paragraphs, and larger portions 
of the text (e.g., a section, chapter, scene, or stanza) relate to each other and the whole. 

6. Assess how point of view or purpose shapes the content and style of a text. 
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Table 11. Summary of Reading Standards for Literature Coding (Part 2) 


Literature 

Standard 

Eligible Item Types 

Comments* 

DOK Range** 

RL.x.4 

SR, ECR, TE, PT 

305 

1-3 

RL.x.5 

SR, ECR, TE, PT 

308 (grades 3, 4, 8-12) 

1- 3 (grades 3, 4) 

2- 3 (grades 5, 6) 

3- 4 (grades 7-12) 

RL.x.6 

SR (grades 4-12), 
ECR, TE, PT 

308 (grade 4) 

2-3 (grade 3) 

3 (grade 4) 

2- 4 (grades 5, 6) 

3- 4 (grades 7-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


Table 12. Summary of Reading Standards for Informational Text Coding (Part 2) 


Informational 

Text 

Standard 

Eligible Item Types 

Comments* 

DOK Range** 

Rl.x.4 

SR, ECR, TE, PT 

305 

1-3 

Rl.x.5 

SR, ECR, TE, PT 

308 (grades 5, 7-12) 

1-2 (grade 3) 

1- 3 (grade 4) 

2- 4 (grades 5, 11-12) 
2-3 (grades 6-10) 

Rl.x.6 

SR (grades 4-12), 
ECR, TE, PT 

308 (grades 4, 7-12) 

2- 3 (grades 3, 6-8) 

3- 4 (grade 4) 

2-4 (grades 5, 9-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


All standards are eligible for assessment via SR items except for RL.3.6 and Rl.3.6, which require 
students to "distinguish their own point of view" from that of "the narrator or those of the characters" 
or "of the author of a text." All standards are eligible for assessment via the other three proposed item 
types at all grade levels. For several grade-level standards, analysts noted that "full coverage of the 
standard may not be possible with selected response" (comment code 308). 

Analysts noted that content in RL.x.4 and Rl.x.4 "overlaps with standard(s) in another strand" (comment 
code 305). These standards address determining the meaning of words and phrases, content that is also 
addressed by Language standards L.x.4, L.x.5, and L.x.6. 

Integration of Knowledge and Ideas 

7. Integrate and evaluate content presented in diverse media and formats, including visually and 
quantitatively, as well as in words. 

8. Analyze the structure of texts, including how specific sentences, paragraphs, and larger portions 
of the text (e.g., a section, chapter, scene, or stanza) relate to each other and the whole. (Not 
applicable to literature) 

9. Analyze how two or more texts address similar themes or topics in order to build knowledge or 
to compare the approaches the authors take. 
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Table 13. Summary of Reading Standards for Literature Coding (Part 3) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

RL.x.7 

SR, ECR, TE, PT 

308 (grades 3, 6-12) 
312 (grades 4-12) 

2-3 (grade 3) 

2- 4 (grades 4, 5) 

3- 4 (grades 6-12) 

RL.x.9 

SR, ECR, TE, PT 

308 (grades 3-10) 

3-4 (grades 3, 4, 6-12) 
1-2 (grade 5) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


Table 14. Summary of Reading Standards for Informational Text Coding (Part 3) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

Rl.x.7 

SR (grades 3-5, 7-12), 
ECR, TE, PT 

308 (grades 3, 4, 7-12) 
313 (grades 5, 6) 

312 (grades 6-12) 

2- 3 (grades 3, 4) 
1-3 (grade 5) 

3- 4 (grades 6-12) 

Rl.x.8 

SR, ECR, TE, PT 

308 (grades 5-12) 
301 (grades 6-12) 

2 (grade 3) 

2-3 (grades 4, 6) 

2-4 (grades 5, 7-12) 

Rl.x.9 

SR (grades 3, 6-12), 
ECR, TE, PT 

308 (grades 3, 6-12) 
313 (grades 4, 5) 

3 (grade 3) 

3-4 (grades 4, 6-10) 
2-4 (grade 5) 

2-3 (grades 11-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


For the literature standards, all four item types proposed for the reading assessment are eligible item 
types for both standards at all grades. For the informational text standards, all standards are eligible for 
assessment via SR items except for Rl.6.7, Rl.4.9, and Rl.5.9, since assessing these standards would 
involve measuring the student's ability to "integrate information" and "develop a coherent 
understanding of a topic or issue" or to write or speak about a topic "knowledgeably." All informational 
text standards are eligible for assessment via the other three proposed item types at all grade levels. 

Analysts noted that for many of the grade-level standards, "full coverage of the standard may not be 
possible with selected response" (comment code 308). In addition, for RL.x.7 at grades 4-12 and Rl.x.7 
at grades 6-12, they noted that "technological enhancement may be necessary for all item types for this 
standard" (comment code 312). Finally, for Rl.5.7, Rl.6.7, Rl.4.9, and Rl.5.9, analysts noted that "the 
parameters for assessment may be difficult to define for a component of this standard" (comment code 
313). 

Range of Reading and Level of Text Complexity 

10. Read and comprehend complex literary and informational texts independently and proficiently. 

Standard RL.x.lO/Rl.x.lO is not eligible for assessment via the proposed item types, as this standard 
does not describe assessable content per se but instead "informs the complexity of text appropriate for 
assessment" (comment code 309). 
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Reading Standards: Foundational Skills, Grades 3-5 


Phonics and Word Recognition 

1. Know and apply grade-level phonics and word analysis skills in decoding words. 

Fluency 

2. Read with sufficient accuracy and fluency to support comprehension. 

Almost all of the grade-level standards in this strand are not eligible for summative assessment via the 
item types proposed for the reading assessment, as the skills assessed by these standards are "best 
measured by in-person observation" (comment code 302). A summative assessment may be best used 
to assess higher-level integrated skills, and decoding and fluency, which are more basic and isolated 
skills, may be best measured in person as part of an interim or formative assessment. Standard RF.3.3 
(DOK 1) is the only eligible standard, assessable via SR, TE, and PT, though analysts noted that "a portion 
of the standard is not measurable via on-demand assessment" (comment code 307). 

Writing Standards. Grades 3-12 

Text Types and Purposes 

1. Write arguments to support claims in an analysis of substantive topics or texts, using valid 
reasoning and relevant and sufficient evidence. 

2. Write informative/explanatory texts to examine and convey complex ideas and information 
clearly and accurately through the effective selection, organization, and analysis of content. 

3. Write narratives to develop real or imagined experiences or events using effective technique, 
well-chosen details, and well-structured event sequences. 


Table 15. Summary of Writing Standards Coding (Part 1) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

W.x.l 

TE, PT, WP 

306 

3-4 

W.x.2 

TE, PT, WP 

306 

3-4 

W.x.3 

TE, PT, WP 

306 

3-4 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


All of the standards are eligible for assessment via the open-ended item types proposed for the writing 
assessment. Analysts noted that the standards overlap with another standard in this strand (comment 
code 306) since the writing process involves integrating the skills described in the standards addressing 
text types and purposes with standard W.x.4, which addresses production of writing (see below). 

Production and Distribution of Writing 

4. Produce clear and coherent writing in which the development, organization, and style are 
appropriate to task, purpose, and audience. 

5. Develop and strengthen writing as needed by planning, revising, editing, rewriting, or trying a 
new approach. 
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Table 16. Summary of Writing Standards Coding (Part 2) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

W.x.4 

TE, PT, WP 

306 

3-4 

W.x.5 

SR, TE, PT, WP 

303 (grades 3-8) 

304 

305 

301 (grades 5-12) 

1-4 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


All of the standards are eligible for assessment via the open-ended item types proposed for the writing 
assessment. While W.x.5 is also assessable via SR, analysts noted that for grades 5 through 12, "at least 
one verb in the standard requires the student to generate a response" (comment code 301). Analysts 
also noted that at least a portion of this standard "requires guidance and support from an adult" 
(comment code 303) in grades 3 through 8, and "requires interaction with others" (comment code 304) 
at all grades. In addition, the standard overlaps with standards in another strand (comment code 305), 
specifically Language standards 1-3. 

6. Use technology, including the Internet, to produce and publish writing and to interact and 
collaborate with others. 

Standard W.x.6 is not eligible for assessment via the proposed item types, as this standard does not 
describe assessable content per se but instead "is related to process rather than outcome" (comment 
code 310). In addition, the standard "requires guidance and support from an adult" (comment code 303) 
in grades 3 through 5. 

Research to Build and Present Knowledge 

7. Conduct short as well as more sustained research projects based on focused questions, 
demonstrating understanding of the subject under investigation. 

8. Gather relevant information from multiple print and digital sources, assess the credibility and 
accuracy of each source, and integrate the information while avoiding plagiarism. 

9. Draw evidence from literary or informational texts to support analysis, reflection, and research. 


Table 17. Summary of Writing Standards Coding (Part 3) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

W.x.7 

TE, PT, WP 
(grades 6-12) 

310 (grades 3-5) 
307 (grades 6-12) 

311 (grades 6-12) 
313 (grades 6-12) 

3-4 (grade 6) 

2-4 (grades 7-12) 

W.x.8 

SR (grades 6-12), 
TE, PT 

307 (grades 3-5) 
301 (grades 6-12) 

308 (grades 6-7) 
313 (grades 7-12) 

1-2 (grades 3-5) 
1-4 (grades 6-12) 

W.x.9 

(begins in grade 4) 

TE, PT, WP 

305 (grades 4-12) 

2-3 (grades 4-5) 
2-4 (grades 6-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 
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Standard W.x.7 is not eligible for on-demand summative assessment in grades 3 through 5, as the skills 
are "related to process rather than outcome" (comment code 310). Portions of this standard are 
assessable via all eligible open-ended item types for writing in grades 6 through 12, though analysts 
noted that full coverage of the standard "requires an extended amount of time" (comment code 311), 
and that portions of the standard may be difficult to assess via on-demand tasks (comment codes 307 
and 313). Standard W.x.8 is assessable via SR from grades 6 through 12, and via TE and PT at all grades, 
though analysts noted that full coverage of the standard may not be possible with SR (comment codes 
301 and 308) and that, as with W.x.7, portions of the standard may be difficult to assess via on-demand 
tasks (codes 307, 313). Standard W.x.9 is assessable via open-ended item types for writing and overlaps 
with standards in another strand (code 305), specifically Reading. 

Range of Writing 

10. Write routinely over extended time frames (time for research, reflection, and revision) and 
shorter time frames (a single sitting or a day or two) for a range of tasks, purposes, and 
audiences. 

Standard W.x.10 is not eligible for assessment via the proposed item types, as this standard is "related 
to process rather than outcome" (comment code 310). In this case, the skill described is clear; however, 
it is not a content skill that can be measured via an on-demand summative assessment with the item 
types proposed by the Consortium for its summative assessment. 

Speaking and Listening Standards. Grades 3-12 

Comprehension and Collaboration 

1. Prepare for and participate effectively in a range of conversations and collaborations with 
diverse partners, building on others' ideas and expressing their own clearly and persuasively. 

This standard is not eligible for on-demand summative assessment, as performing the skills requires 
interaction with others (comment code 304) and requires an extended amount of time (comment code 
311). 

2. Integrate and evaluate information presented in diverse media and formats, including visually, 
quantitatively, and orally. 

3. Evaluate a speaker's point of view, reasoning, and use of evidence and rhetoric. 


Table 18. Summary of Speaking and Listening Standards Coding (Part 1) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

SL.x.2 

TE 

314 (grades 5-12) 
313 (grades 11-12) 

1- 2 (grade 3) 

2- 3 (grades 4, 5) 

2- 4 (grades 6-10) 

3- 4 (grades 11-12) 

SL.x.3 

TE 

314 (grades 3-8) 

1- 2 (grades 3, 4) 

2- 3 (grades 5, 6) 
2-4 (grades 7-12) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 
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Both standards are eligible via TE items 11 . Analysts noted that, for many grade-level standards, "only 
listening would be measured if responses were written" (comment code 314), and that, for SL.9-10.2 
and SL. 11-12. 2, "the parameters for assessment may be difficult to define for a component of this 
standard" (comment code 313). 

Presentation of Knowledge and Ideas 

4. Present information, findings, and supporting evidence such that listeners can follow the line of 
reasoning and the organization, development, and style are appropriate to task, purpose, and 
audience. 

5. Make strategic use of digital media and visual displays of data to express information and 
enhance understanding of presentations. 

6. Adapt speech to a variety of contexts and communicative tasks, demonstrating command of 
formal English when indicated or appropriate. 


Table 19. Summary of Speaking and Listening Standards Coding (Part 2) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

SL.x.4 

TE, OR 


1-4 (grade 3) 

1-3 (grades 4-12) 

SL.x.5 

TE, OR 

307 (grade 3) 

313 (grades 9-12) 

2-3 

SL.x.6 

TE, OR 

307 (grade 3) 
305 

1-2 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


All three standards are assessable via TE items, with oral response required. Analysts noted that for 
some of the grade-level standards, a portion of the standard "is not measurable via on-demand 
assessment" (comment code 307) or "the parameters for assessment may be difficult to define for a 
component of this standard" (comment code 313). 

Language Standards. Grades 3-12 

Conventions of Standard English 

1. Demonstrate command of the conventions of standard English grammar and usage when 
writing or speaking. 

2. Demonstrate command of the conventions of standard English capitalization, punctuation, and 
spelling when writing. 

Knowledge of Language 

3. Apply knowledge of language to understand how language functions in different contexts, to 
make effective choices for meaning or style, and to comprehend more fully when reading or 
listening. 


11 As presented in the SBAC proposal, all Speaking and Listening standards are assessed via TE items. 
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Table 20. Summary of Language Standards Coding (Part 1) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

L.x.l 

SR, TE, PT, WP 

301 

1-2 (grades 3-5, 7- 



305 

12) 



308 

1 (grade 6) 

L.x.2 

SR, TE, PT, WP 

301 

1 



305 




308 


L.x.3 

SR, TE, PT, WP 

301 

2 (grades 3, 4) 



305 

2-4 (grade 5) 



308 

2-3 (grade 6) 

1-3 (grades 7, 9-12) 
3 (grade 8) 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 


All three standards are assessable via all eligible item types for the Language strand, though analysts 
noted that "full coverage of the standard may not be possible with selected response" (comment code 
308) and that "at least one verb in the standard requires the student to generate a response" (comment 
code 301). These standards overlap with standards in another strand (comment code 305), specifically 
Reading, Writing, and Speaking and Listening standards. 

Vocabulary Acquisition and Use 

4. Determine or clarify the meaning of unknown and multiple-meaning words and phrases by using 
context clues, analyzing meaningful word parts, and consulting general and specialized 
reference materials, as appropriate. 

5. Demonstrate understanding of figurative language, word relationships, and nuances in word 
meanings. 

6. Acquire and use accurately a range of general academic and domain-specific words and phrases 
sufficient for reading, writing, speaking, and listening at the college and career readiness level; 
demonstrate independence in gathering vocabulary knowledge when encountering an unknown 
term important to comprehension or expression. 


Table 21. Summary of Language Standards Coding (Part 2) 


Standard 

Eligible Item Types 

Comments* 

DOK Range** 

L.x.4 

SR, TE, PT 

305 

307 (grades 6-12) 

1-3 (grades 3-5) 
1-2 (grades 6-12) 

L.x.5 

SR, TE, PT 

305 

307 (grades 6-12) 

1- 3 (grades 3-8) 

2- 3 (grades 9-12) 

L.x.6 

SR, TE, PT, WP 

305 

307 (grades 4-12) 
313 (grades 6-12) 

1-2 


*See Appendix D for an explanation of the comments. 
**For grades 3-12 unless otherwise noted. 
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All three standards are assessable via SR, TE, and PT item types, and L.x.6 is also assessable via WP. 
Analysts noted that, at some grade levels, portions of the standard are "not measurable via on-demand 
assessment" (comment code 307) and "the parameters for assessment may be difficult to define for a 
component of this standard" (comment code 313). These standards overlap with standards in another 
strand (comment code 305), specifically Reading, Writing, and Speaking and Listening standards. 
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Mathematics Findings 


This section presents a summary of the findings for the eligible content analysis in mathematics. General 
findings for the three eligibility criteria are presented first, followed by depth of knowledge results and 
specific findings related to item types and depth of knowledge. 

Eligibility 

As described in detail in the Methodology section of this report, standards were considered eligible for 
the summative assessment if they were judged to be learnable during the school year, expected of all 
students, and measurable via on-demand assessment. For high school mathematics, the "expected of all 
students" criterion focused on whether the standard was labeled with a (+), indicating additional 
mathematics for advanced course preparation. As described elsewhere in this subsection, standards 
marked with a (+) were coded as not expected of all students, and therefore are not eligible for the 
summative assessment for accountability purposes. 

Table 22 shows the number of standards in each grade and conceptual category that were judged to be 
eligible or not eligible, as determined by the three criteria for eligibility. 


Table 22. Number of Mathematics CCSS Eligible for Summative Assessment 


Grade or 

Conceptual Category 

Total 

Learnable 

Expected 

Measurable 

Eligible 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

3 

25 

25 

0 

25 

0 

25 

0 

25 

0 

4 

28 

28 

0 

28 

0 

28 

0 

28 

0 

5 

26 

26 

0 

26 

0 

26 

0 

26 

0 

6 

29 

29 

0 

29 

0 

29 

0 

29 

0 

7 

24 

24 

0 

24 

0 

24 

0 

24 

0 

8 

28 

28 

0 

28 

0 

27 

1 

27 

1 

Number and Quantity 

27 

27 

0 

9 

18 

27 

0 

9 

18 

Algebra 

27 

27 

0 

23 

4 

27 

0 

23 

4 

Functions 

28 

28 

0 

22 

6 

28 

0 

22 

6 

Geometry 

43 

43 

0 

37 

6 

41 

2 

35 

8 

Statistics and Probability 

31 

31 

0 

22 

9 

31 

0 

22 

9 

TOTAL 

316 

316 

0 

273 

43 

313 

3 

270 

46 

Percent of Total 

100% 

0% 

86% 

14% 

99% 

1% 

85% 

15% 


Of the 316 grade-level and conceptual category-level standards in grades 3-8 and high school, 270 were 
judged to be eligible for the summative assessment; 46 were not eligible. The eligibility was distributed 
across the grades as follows: 

• In grades 3-7, all standards were eligible. 

• In grade 8, 27 of 28 standards were eligible. One grade 8 standard in the Geometry domain was 
ineligible; this standard was judged to be not measurable via on-demand assessment. 

• In high school, 111 of 156 standards were eligible and 45 were not eligible. 

• Of the 45 ineligible high school standards, 43 were not expected of all students. The largest 
number of standards not expected of all students was in the Number and Quantity conceptual 
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category, but every conceptual category included some standards not expected of all students. 
The remaining two ineligible standards, both in Geometry, were judged to be not measurable 
via on-demand assessment and more appropriate for classroom assessment. 

Learnable During the School Year 

All 316 mathematics standards in grades 3-8 and high school were judged to be learnable during the 
school year. 

Expectations of All Students 

All standards in grades 3-8 were judged to be expected of all students. In high school, 43 of 156 
standards were coded as not expected of all students, due to being designated with a (+). According to 
the CCSS (CCSSO and NGA, 2010b, p. 57), the mathematics CCSS for high school "specify the 
mathematics that all students should study in order to be college and career ready," and certain 
standards designated with a (+) describe "additional mathematics that students should learn in order to 
take advanced courses such as calculus, advanced statistics, or discrete mathematics." Thus, all 
standards were judged to be expected of all students, except for those standards designated with a (+). 
The Number and Quantity conceptual category had the greatest number of standards judged to be not 
expected of all students (18 of 27). Statistics and Probability had 9 of 31, Functions had 6 of 28, 
Geometry had 6 of 43, and Algebra had 4 of 27. 

Additionally, in three Functions standards (F.IF.7, F.BF.l, and F.BF.4), only a portion of the standard is 
designated with a (+). These standards were judged to be expected of all students but were coded with 
comment code 202, "A portion of this standard is labeled with a (+), indicating that the content should 
be learned by students in order to take advanced courses." 

Measurable Via On-Demand Assessment 

All standards in grades 3-7 were judged to be measurable via on-demand summative assessment using 
at least some of the item types defined in the SBAC proposal. In grade 8, 27 of 28 standards were judged 
to be measurable via on-demand summative assessment. One grade 8 standard in the Geometry 
domain was judged to be not measurable via on-demand summative assessment. The standard, 8.G.1 
("Verify experimentally the properties of rotations, reflections, and translations . . . "), was judged to be 
more appropriate for classroom assessment. 12 

In high school, all standards in all conceptual categories except Geometry were judged to be measurable 
via on-demand summative assessment. In Geometry, two of 43 standards were judged to not be 
measurable via on-demand assessment. One standard, G.CO.4 ("Develop definitions of rotations, 
reflections, and translations . . . "), was judged to focus on process rather than outcome. The other 
standard, G.SRT.l ("Verify experimentally the properties of dilations given by a center and a scale 
factor"), was judged to be more appropriate for classroom assessment. 


12 As noted previously, outcomes of this analysis are intended as a starting point to inform discussion and should in 
no way be interpreted to constrain the Consortium's discussion and decisions. For example, the analysts of this 
study judged standard 8.G.1 more appropriate for classroom assessment than for summative assessment. 

Flowever, there are numerous considerations to be made when designing the summative assessment (e.g., content 
coverage). Therefore, with further discussion by Consortium groups (e.g., Test Design Work Group, content 
specifications committee), it may be determined that some standards that were judged by this study's analysts as 
more appropriate for classroom assessment should be included on the Consortium's summative assessment. 
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Measurability by Item Type 

All standards determined to be measurable via on-demand summative assessment were then coded for 
the types that could be most efficiently used to assess those standards. In mathematics, the types of 
items included in the analysis were selected-response items, extended constructed-response items, 
technology-enhanced items, and performance tasks. This rating does not indicate that one item type 
could necessarily assess the full depth and breadth of the standard, but instead indicates that at least a 
component of the standard is assessable using the item type. Table 23 shows, for the eligible standards, 
the item types by which the standards were judged to be measurable. 


Table 23. Item Types by Which Eligible Mathematics Standards Were Judged to Be Measurable 


Grade or 

Conceptual Category 

Total 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

3 

25 

24 

1 

25 

0 

25 

0 

25 

0 

4 

28 

28 

0 

28 

0 

28 

0 

28 

0 

5 

26 

26 

0 

26 

0 

26 

0 

26 

0 

6 

29 

29 

0 

29 

0 

29 

0 

29 

0 

7 

24 

23 

1 

24 

0 

24 

0 

24 

0 

8 

27 

27 

0 

27 

0 

27 

0 

27 

0 

Number and Quantity 

9 

8 

1 

9 

0 

9 

0 

9 

0 

Algebra 

23 

22 

1 

23 

0 

23 

0 

23 

0 

Functions 

22 

21 

1 

22 

0 

22 

0 

22 

0 

Geometry 

35 

23 

12 

35 

0 

35 

0 

35 

0 

Statistics and Probability 

22 

21 

1 

22 

0 

22 

0 

22 

0 

TOTAL 

270 

252 

18 

270 

0 

270 

0 

270 

0 

Percent of Total 

93% 

7% 

100% 

0% 

100% 

0% 

100% 

0% 


Of the 270 eligible standards, 252 were judged to be assessable via all four item types. All 270 standards 
were judged to be assessable via extended constructed-response items, technology-enhanced items, 
and performance tasks. Eighteen standards were judged to be not measurable via selected-response 
items. Of the 18 standards, one was in grade 3, one was in grade 7, and the remaining 16 were in high 
school. Twelve of those high school standards were in the Geometry conceptual category. 
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Response Type 

No mathematics standard was judged to require an oral response for assessment. 

Comments 

Two comments were used most frequently to expand on the measurability criterion: 301 and 308. 

• Comment code 301: "At least one verb in the standard requires the student to generate a 
response." This comment was assigned to 87 eligible standards (97 total standards). 

• Comment code 308: "Full coverage of the standard may not be possible with selected 
response." This comment was assigned to 93 eligible standards (103 total standards). 

Both comments were associated with standards across all grades and conceptual categories. The two 
comments were often associated with the same standards. 

Eight standards, all eligible, were judged to have at least a portion of the standard that would be best 
measured via in-person observation (comment code 302). These standards were from grades 3, 4, 5, and 
7, and the high school conceptual category Geometry. The standards that received these comments had 
elements such as estimation, mental computation, and use of tools, which may be difficult to assess 
using an on-demand assessment. 

Six standards, all eligible, were flagged as being possible for use "as a stand-alone for a performance 
task" (comment code 315). These standards were from grade 7 and the high school conceptual 
categories Statistics and Probability and Geometry. The four grade 7 standards were in the Statistics and 
Probability conceptual category, with three standards under the cluster "Investigate chance processes 
and develop, use, and evaluate probability models," and one standard under the cluster "Use random 
sampling to draw inferences about a population." The standards receiving this comment tend to be 
broader than other standards. While many standards (including these) could be used in combination 
with each other to create a meaningful performance task, these six standards also appear to have 
sufficient content individually to support a performance task that includes complex thinking in the 
context of an extended task. These standards would benefit from the extended amount of time offered 
to complete a performance task. For example, 7.SP.8c, "Design and use a simulation to generate 
frequencies for compound events," requires time for students to complete a related task with fidelity to 
the intent of the standard. While other item types could also assess a portion of this substandard, none 
are likely to offer the opportunity to truly explore the full breadth of the intention of the standard. 

Three standards, two of which were eligible, were noted as having at least a portion of the standard 
related to process rather than outcome (comment code 310). The two eligible standards, in grade 8 
(8. EE. 8) and the high school conceptual category Algebra (A.APR.6), both specify solving by inspection, 
which was determined to be a key focus of the standards. 

Depth of Knowledge 

As described in the Methodology section of this report, each standard, regardless of its eligibility rating, 
was analyzed to determine the range of depth of knowledge levels in the standard. Table 24 shows, for 
all standards, the number of standards to which each depth of knowledge level was assigned. The 
percentages of total standards at each DOK level sum to above 100 since a standard could be rated for 
multiple DOK levels. 
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Table 24. Depth of Knowledge Levels of All Mathematics Standards 


Grade or Conceptual Category 

Total 

Depth of Knowledge Level 

1 

2 

3 

4 

3 

25 

24 

24 

3 

0 

4 

28 

28 

16 

5 

0 

5 

26 

26 

18 

3 

0 

6 

29 

29 

20 

2 

0 

7 

24 

18 

22 

8 

0 

8 

28 

26 

25 

9 

0 

Number and Quantity 

27 

27 

15 

0 

0 

Algebra 

27 

26 

21 

7 

0 

Functions 

28 

27 

24 

4 

0 

Geometry 

43 

24 

36 

19 

1 

Statistics and Probability 

31 

27 

29 

7 

0 

TOTAL 

316 

282 

250 

67 

1 

Percentage of Total Standards at DOK Level 

(Standards may cover a range of DOK levels) 

89% 

79% 

21% 

< 1% 


Across all grades and conceptual categories, the majority of standards were coded to DOK Level 1 
and/or Level 2. In grade 7, grade 8, and especially the high school conceptual category Geometry, a 
notable number of standards were also coded to Level 3. One standard in Geometry was coded to 
Level 4. 

Since a standard could be coded to more than one DOK level, results for depth of knowledge can also be 
examined by looking at the range of depth of knowledge of the standards. Table 25 shows, for the 
eligible standards only, the number of standards in each grade and conceptual category with each range 
of depth of knowledge. A standard with a range of 1-1 was coded to Level 1 only. A standard with a 
range of 1-3 was coded to Level 1, Level 2, and Level 3. 


Table 25. Range of Depth of Knowledge of Eligible Mathematics Content Standards 


Grade or 

Conceptual Category 

Total 

Range of Depth of Knowledge 

1-1 

1-2 

1-3 

2-2 

2-3 

2-4 

3-3 

3 

25 

1 

20 

3 

1 

0 

0 

0 

4 

28 

12 

11 

5 

0 

0 

0 

0 

5 

26 

8 

15 

3 

0 

0 

0 

0 

6 

29 

9 

18 

2 

0 

0 

0 

0 

7 

24 

2 

13 

3 

1 

5 

0 

0 

8 

27 

3 

15 

8 

0 

1 

0 

0 

Number and Quantity 

9 

4 

5 

0 

0 

0 

0 

0 

Algebra 

23 

4 

12 

6 

0 

1 

0 

0 
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Grade or 

Conceptual Category 

Total 

Range of Depth of Knowledge 

1-1 

1-2 

1-3 

2-2 

2-3 

2-4 

3-3 

Functions 

22 

4 

16 

2 

0 

0 

0 

0 

Geometry 

35 

1 

16 

4 

2 

5 

1 

6 

Statistics and Probability 

22 

2 

16 

1 

1 

2 

0 

0 

TOTAL 

270 

50 

157 

37 

5 

14 

1 

6 


In all grades except grade 4, among eligible standards, the greatest number of standards had a range of 
depth of knowledge of 1-2. In grade 4, the greatest number of standards (12) had a range of depth of 
knowledge of 1-1, while 11 standards had a range of depth of knowledge of 1-2. All grades and 
conceptual categories included some eligible standards with a range that included Level 3, except for 
Number and Quantity. Some Number and Quantity standards judged to be ineligible also had a range of 
depth of knowledge of 1-1 and 1-2. Geometry was the only conceptual category that had standards 
with a range of 3-3, and the only conceptual category with a standard coded to Level 4. 

It should be noted that while the majority of the mathematics content knowledge as described by the 
Standards for Mathematical Content was rated at DOK Levels 1 and 2, it is not necessarily the case that 
assessment items would be developed at the same DOK levels. Developers of the summative 
assessment items/tasks may decide to require students to apply the Standards for Mathematical 
Practice in varied and complex ways, resulting in assessment items/tasks at DOK Levels 3 or 4. 

Standards Not Measurable Via Selected-Response Items 

As noted earlier in this section, most standards in the mathematics CCSS (252 of the 270 eligible 
standards) were found to be eligible for assessment by all four item types. Eighteen eligible standards, 
all but two of which were in high school, were judged to not be measurable via selected-response items. 
Of those 18 standards, 17 had a range of depth of knowledge that included Level 2 and/or Level 3, while 
three standards had a range of depth of knowledge that included Level 1. Of those three, one had a 
depth of knowledge of Level 1 only. Two of the three standards were noted as having portions of the 
standard that would be best measured by in-person observation (comment code 302), and one was 
flagged as a possible standard for a stand-alone performance task (comment code 315). 

The majority of standards not measurable via selected-response items (13 of the 18) were in the domain 
or conceptual category of Geometry. These standards included the skills of proofs, constructions, and 
giving an informal argument; analysts judged that selected-response items would not adequately 
measure these skills, as they require the student to generate a response. 

Nearly one-third of the standards (93 eligible; 103 total) received the comment that full coverage of the 
standard may not be possible with selected-response items. These standards received comment code 
308 to indicate that, while selected-response items could be used to assess portions of the standard, at 
least one component of the standard would require at least one other item type. The standards 
receiving this code occurred across all grades and conceptual categories, and with the following ranges 
of depth of knowledge: 1-1, 1-2, 1-3, and 2-3. These standards often also received the comment that 
at least one verb in the standard required the student to generate a response (comment code 301). 
Analysts' judgments were based on the specific wording of the standards, and if the language of a 
standard described a student response, rather than selecting an answer, this language was interpreted 
literally. For example, in the grade 7 standard 7. NS. 2d (". . . Convert a rational number to a decimal using 
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long division; know that the decimal form of a rational number terminates in Os or eventually repeats"), 
the specific mention of "using long division" was interpreted as requiring a response that could not be 
assessed via selected-response item. As another example, in grade 4, 4.NBT.5 requires that students 
"illustrate and explain the calculation by using equations, rectangular arrays, and/or area models." A 
selected-response item in which a student selects an illustration or explanation was interpreted as not 
addressing the specific skills of actually illustrating and explaining, as included in the standard. 

Eligible Standards Coded to Depth of Knowledge Level 1 Only 

A total of 50 eligible standards were judged to have a range of depth of knowledge of Level 1 only. Of 
those 50, all were coded as assessable via all four item types, except for one item that was not 
assessable via selected response. These Level 1 standards occurred in all grades and conceptual 
categories. In many cases, a selected-response item may be sufficient to assess the content of these 
Level 1 standards, or the standards may be clustered or integrated into open-ended item types. It is 
therefore not necessarily the case that assessment items would be developed at the same DOK Level 1. 
As suggested previously, developers of the summative assessment items/tasks may decide to require 
students to apply the Standards for Mathematical Practice in varied and complex ways, resulting in 
assessment items/tasks at DOK Levels 3 or 4. 

Eight of the 50 standards received comment codes 301 and 308, indicating that a portion of the 
standard required students to generate a response and that full coverage of the standard may not be 
possible via selected-response items. Thus, it is important to note that the nature of the tasks of some 
standards, while cognitively at Level 1, may require item types other than selected response to assess 
the skill as written in the standards. For example, in the grade 8 standard 8.EE.1 ("Know and apply the 
properties of integer exponents to generate equivalent numerical expressions"), knowing and applying 
the properties of integer exponents is Level 1 content. A portion of this standard could be assessed via 
selected-response items, but such items would only be able to ask students to "recognize" equivalent 
expressions. Assessing the "generate" portion of the standard would require students to generate the 
equivalent expressions, thus necessitating assessment via an open-ended item type. 

Eligibility and depth of knowledge ratings can also be examined at the individual standard level. 
Analysts' ratings for each standard are included in Appendix E. 
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CONSIDERATIONS FOR NEXT STEPS 


Outcomes of this descriptive analysis are intended as a starting point to inform the Consortium efforts 
to design its summative assessment and develop its test and item specifications. The Consortium's 
further examination and discussion of the study's descriptive data and their implications for the design 
and development of its summative assessment is expected, and this section is intended to help 
members of the Consortium involved in item and test development (e.g., the Item Design, Test 
Development, Technology Approach, and Accessibility and Accommodations Work Groups and the 
content specifications committee) use findings from this analysis to inform their discussions and 
activities. 

Suggested Considerations 
Comment Codes 

As described in the Findings section of this report, many of the standards have comment codes (e.g., 
associated with analyses of measurability and DOK) 13 . These comment codes have potential implications 
for content clustering, item development, test design, test administration, and the use of technology, as 
shown in Table 26 below. 14 


Table 26. Suggested Implications of Comment Codes 


Comment Code 

Implications 

Content 

Clustering 

Item 

Development 

Test 

Design 

Test 

Administration 

Use of 
Technology 

201: Standard is labeled with a (+), 
indicating that the content should 
be learned by students in order to 
take advanced courses. 


V 

V 



202: A portion of this standard is 
labeled with a (+), indicating that 
the content should be learned by 
students in order to take advanced 
courses. 


V 

V 



301: At least one verb in the 
standard requires the student to 
generate a response. 

V 

V 

V 

V 

V 

302: At least a portion of this 
standard is best measured by in- 
person observation. 


V 

V 

V 


303: Assessment of this standard 
(or a portion of this standard) may 
require guidance and support from 
an adult. 


V 


V 

V 

304: Assessment of this standard 


V 


V 

V 


13 As noted in the Methodology section, comment codes were proposed by analysts as they completed their 
ratings; the wording of the codes was then standardized to ensure consistent use across analysts. 

14 These areas of focus (i.e., content clustering, item development, test design, test administration, use of 
technology) were selected based on the 2010-2011 activities reflected in the Consortium's Master Work Plan and 
High-Level Time Line and Work Strands (SBAC, 2010a). 
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Comment Code 

Implications 

Content 

Clustering 

Item 

Development 

Test 

Design 

Test 

Administration 

Use of 
Technology 

(or a portion of this standard) may 
require interaction with others. 






305: This standard (or a portion of 
this standard) overlaps with one or 
more standard(s) in another strand. 

V 

V 

V 



306: This standard (or a portion of 
this standard) overlaps with one or 
more standard(s) in this strand. 

V 

V 

V 



307: A portion of this standard is 
not measurable via on-demand 
assessment. 


V 

V 



308: Full coverage of the standard 
may not be possible with selected 
response. 

V 

V 

V 


V 

309: Statements in this standard 
inform the complexity of text 
appropriate for assessment. 



V 



310: At least a portion of this 
standard is related to process 
rather than outcome. 

V 

V 

V 

V 

V 

311: This standard (or a portion of 
this standard) requires an extended 
amount of time. 


V 

V 

V 

V 

312: Technological enhancement 
may be necessary for all item types 
for this standard. 

V 

V 

V 


V 

313: The parameters for 
assessment may be difficult to 
define for a component of this 
standard. 

V 

V 

V 



314: Only listening would be 
measured if responses were 
written. 

V 

V 

V 

V 


315: This standard may be used as a 
stand-alone for a performance task. 


V 

V 

V 

V 

401: Higher DOK applies only to a 
portion of the standard that is to be 
learned by students in order to take 
advanced courses. 

V 

V 

V 




It is recommended that users of the results of this analysis review and discuss the specific standards 
with comment codes relevant to their task (see the Eligible Content Data Workbook in Appendix E). For 
example, those involved in defining the technology approach for the Consortium's summative 
assessment should review standards with comment codes 301, 303, 304, 308, 310, 311, 312, and 315, 
and discussion of the implications for the design and specifications of the item authoring application and 
scoring engine should be coordinated with those involved in item design, test design, test 
administration, and accessibility and accommodations as appropriate. Cross-work group coordination is 
critical to help ensure a cohesive and coherent assessment. 
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It is also critical to underscore that the information presented in this report is a starting point for the 
Consortium's efforts related to test and item development. The Consortium should use this study's data 
and recommendations to inform its further explication of the content to be assessed; that is, in terms of 
what is to be measured, what is not to be measured, how to best measure the content, and how to 
ensure accessibility to all students of the content measured. Subsequent discussions should focus on 
issues such as whether and how assessable content can be reasonably clustered to reduce the number 
of assessed constructs— for example, which content could and should be clustered, prioritization of the 
standards/clusters to be assessed, and the cognitive demands and types of evidence required by the 
standards/clusters to be assessed. These discussions should occur within the context of understanding 
the purpose of the summative assessment and what the Consortium hopes to accomplish with this 
assessment in terms of student learning and achievement. 

Standards Not Measurable Via Selected-Response Items 

In addition to using the information in Table 26, those involved in item and test design and development 
should systematically examine and discuss those standards determined to be eligible for the summative 
assessment but not measurable via the selected-response item type (i.e., 92 standards in ELA; 

18 standards in mathematics). Consortium work groups and committees (e.g., Item Development, Test 
Design, Technology Approach, Test Administration, Reporting, and Accessibility and Accommodations 
Work Groups and the content specifications committee) should further examine these standards and 
determine, for example, whether and how these standards should be clustered or integrated into open- 
ended item types. In the case of the mathematics standards not measurable with selected-response 
items, work groups and committees may also wish to integrate these standards with the Standards for 
Mathematical Practice to develop assessment items/tasks that measure such clusters of standards in 
varied and complex ways. 

Standards Judged as More Appropriate for Classroom Assessment 

As noted previously, outcomes of this analysis are intended as a starting point to inform discussion and 
should in no way be interpreted to constrain the Consortium's discussion and decisions. With that said, 
some standards (e.g., mathematics standard 8.G.1) were judged to be more appropriate for classroom 
assessment than for summative assessment. However, there are numerous considerations to be made 
when designing the summative assessment (e.g., content coverage). Therefore, further discussion 
should occur among Consortium groups (e.g., Test Design Work Group, content specifications 
committee), to verify exclusion or to determine the need for inclusion on the summative assessment of 
standards judged by this study's analysts as more appropriate for classroom assessment. 

Vertical Alignment of ELA Standards 

As noted in the Introduction to this report, the standards for ELA and mathematics have different 
organizational structures. The ELA standards are organized based on cross-grade College and Career 
Readiness (CCR) Anchor Standards, whereas the mathematics standards are organized based on 
domains that vary across grade levels/spans according to grade-appropriate content. This subsection 
provides additional information on the organization of the ELA standards that may be useful in 
consideration of the results of this study for clustering content and determining learning progressions 
across grade levels. 
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Table 27 provides an example of the relationship between the CCR Anchor Standards, which define 
"general, cross-disciplinary literacy expectations that must be met for students to be prepared to enter 
college and workforce training programs ready to succeed," and the grade-level standards, which define 
"end-of-year expectations and a cumulative progression designed to enable students to meeting college 
and career readiness expectations no later than the end of high school" (CCSSO and NGA, 2010a, p. 4), 
for each strand. The codes for the standards were determined as follows: "Individual CCR anchor 
standards can be identified by their strand, CCR status, and number (R.CCR.6, for example). Individual 
grade-specific standards can be identified by their strand, grade, and number (or number and letter 15 , 
where applicable), so that Rl.4.3, for example, stands for Reading, Informational Text, grade 4, standard 
3 and W.5.1a stands for Writing, grade 5, standard la" (p. 8). 


Table 27. Sample Set of College and Career Readiness Anchor Standards across Grade Levels 


K-12 Anchor 
Standard 

Grade 5 Standard 

Grade Span 11-12 Standard 

College and Career 
Readiness Anchor 
Standards for 
Reading 

Reading Standards 
for Literature K-5 

Reading Standards 
for Informational 
Text K-5 

Reading Standards 
for Literature 6-12 

Reading Standards 
for Informational 
Text 6-12 

R.CCR.2 

Determine central 
ideas or themes of a 
text and analyze their 
development; 
summarize the key 
supporting details 
and ideas. 

RL.5.2 

Determine a theme 
of a story, drama, or 
poem from details in 
the text, including 
how characters in a 
story or drama 
respond to 
challenges or how 
the speaker in a 
poem reflects upon 
a topic; summarize 
the text. 

Rl.5.2 

Determine two or 
more main ideas of 
a text and explain 
how they are 
supported by key 
details; summarize 
the text. 

RL.11-12.2 

Determine two or 
more themes or 
central ideas of a 
text and analyze 
their development 
over the course of 
the text, including 
how they interact 
and build on one 
another to produce 
a complex account; 
provide an 
objective summary 
of the text. 

Rl. 11-12. 2 

Determine two or 
more central ideas 
of a text and analyze 
their development 
over the course of 
the text, including 
how they interact 
and build on one 
another to provide a 
complex analysis; 
provide an objective 
summary of the 
text. 

College and Career 
Readiness Anchor 
Standards for 
Writing 

Writing Standards K-5 

Writing Standards 6-12 

W.CCR.l 

Write arguments to 
support claims in an 
analysis of 
substantive topics or 
texts, using valid 
reasoning and 
relevant and 
sufficient evidence. 

W.5.1 

Write opinion pieces on topics or texts, 
supporting a point of view with reasons and 
information. 

W.ll-12.1 

Write arguments to support claims in an 
analysis of substantive topics or texts, 
using valid reasoning and relevant and 
sufficient evidence. 


For the purposes of this study, lettered content was not coded, though the content was considered as relevant 
to inform understanding of the numbered standard. Lettered details that are included for some of the grade- 
specific standards are not shown in Table 27. 
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K-12 Anchor 
Standard 

Grade 5 Standard 

Grade Span 11-12 Standard 

College and Career 
Readiness Anchor 
Standards for 
Listening and 
Speaking 

Speaking and Listening Standards K-5 

Speaking and Listening Standards 6-12 

SL.CCR.3 

Evaluate a speaker's 
point of view, 
reasoning, and use of 
evidence and 
rhetoric. 

SL.5.3 

Summarize the points a speaker makes and 
explain how each claim is supported by 
reasons and evidence. 

SL.11-12.3 

Evaluate a speaker's point of view, 
reasoning, and use of evidence and 
rhetoric, assessing the stance, premises, 
links among ideas, word choice, points of 
emphasis, and tone used. 

College and Career 
Readiness Standards 
for Language 

Language Standards K-5 

Language Standards 6-12 

L.CCR.5 

Demonstrate 
understanding of 
word relationships 
and nuances in word 
meanings. 

L.5.5 

Demonstrate understanding of figurative 
language, word relationships, and nuances 
in word meanings. 

L.ll-12.5 

Demonstrate understanding of figurative 
language, word relationships, and nuances 
in word meanings. 


Range of Depth of Knowledge 

In some cases for ELA, for example, the DOK ranges for the same anchor standard may vary across grade 
levels. In such cases, special consideration may need to be taken in interpreting the intent of the 
standards in order to represent a progression of skills across grade levels (DOK levels for each grade- 
level standard were determined by analysts based on the particular use of verbs and other relevant 
descriptors in the standard). Such consideration has implications for defining learning progressions 
across grade levels, which will affect assessment design and item development. 
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Additional Examples of Use of Data 

The following examples for ELA and mathematics are intended to help illustrate how to navigate and use 
the information from this analysis (e.g., consideration of the content standards, comment codes, DOK) 
in creating a task. 16,17 

Using Data from Eligible Content Data Workbook to Inform Performance Task Design for ELA: Example 
(Grade 5) 

Conceptual Task: Critically analyze informational text: compare and contrast ideas or concepts in two 
texts and present the analysis in writing in well-organized paragraphs. 

Step 1: Determine set of relevant eligible standards that would be assessed by this task (consider notes 
on measurability; ensure that standard can be assessed with a performance task [PT] item type). 


Standard 

Note 

PT? 

Rl.5.1 

308 

Y 

Rl.5.2 

308 

Y 

Rl.5.3 


Y 

Rl.5.8 

308 

Y 

W.5.2 

306 

Y 

W.5.4 

306 

Y 

W.5.9 

305 

Y 


308: Full coverage of the standard may not be possible with selected response. 
306: Standard overlaps with standard(s) in this strand. 

305: Standard overlaps with standard(s) in another strand. 

Step 2: Consider range of DOK. 


Standard 

Range of DOK 

Rl.5.1 

1-3 

Rl.5.2 

2-3 

Rl.5.3 

1-3 

Rl.5.8 

2-4 

W.5.2 

2-4 

W.5.4 

3-4 

W.5.9 

2-3 


16 The intention of the example tasks— in these cases, performance tasks— is not to present an item prototype or 
to model how best to assess the particular standards referenced in the examples. Rather, the intention is to 
illustrate how to consider the various categories of data (e.g., comment codes, DOK) as Consortium groups (e.g., 
content specifications committee; Item Development and Test Design Work Groups) move forward with their 
design and development activities. The data presented in this report are intended for further discussion by 
Consortium groups. 

17 These examples are also intended to underscore that assessing an integration of knowledge and skills across 
multiple standards should arise from the standards and/or definition of the targeted construct/conceptual task. 
The cognitive demands and types of evidence required by the desired construct also should be defined. 
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Step 3: Consider evidence required for student demonstration of understanding/mastery. 


Standard 

Targeted Level of Cognitive Demand 

Evidence Required 

Rl.5.1 



Rl.5.2 



Rl.5.3 



Rl.5.8 



W.5.2 



W.5.4 



W.5.9 




Step 4: Determine how each standard (or part of the standard) will be assessed by the task and at what 
DOK level. [To be completed as part of development of item specifications and tasks.] 


Standard 

Part of Standard Assessed 

How Task Assesses This Part 
(Include appropriate 
accessibility and 
accommodations strategies) 

DOK Level Required by Task 

Rl.5.1 




Rl.5.2 




Rl.5.3 




Rl.5.8 




W.5.2 




W.5.4 




W.5.9 





Using Data from Eligible Content Data Workbook to Inform Performance Task Design for 
Mathematics: Example (Grade 7) 

Conceptual Task: Students will investigate chance processes and demonstrate the ability to develop, 
use, and evaluate probability models. 

Step 1: Determine set of relevant eligible standards that would be assessed by this task (consider notes 
on measurability; ensure that standard can be assessed with a performance task [PT] item type). 


Standard 

Note 

PT? 

7.SP.6 

308; 315 

Y 

7.SP.7 

301; 308; 315 

Y 

7.SP.8 

301; 308; 315 

Y 


301: At least one verb in the standard requires the student to generate a response. 
308: Full coverage of the standard may not be possible with selected response. 
315: This standard may be used as a stand-alone for a performance task. 
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Step 2: Consider range of DOK. 


Standard 

Range of DOK 

7.SP.6 

2-3 

7.SP.7 

2-3 

7.SP.8 

1-3 


Step 3: Consider evidence required for student demonstration of understanding/mastery. 


Standard 

Targeted Level of Cognitive Demand 

Evidence Required 

7.SP.6 



7.SP.7 



7.SP.8 




Step 4: Determine how each standard (or part of the standard) will be assessed by the task and at what 
DOK level. [To be completed as part of development of item specifications and tasks.] 


Standard 

Part of Standard Assessed 

How Task Assesses This Part 
(Include appropriate 
accessibility and 
accommodations strategies) 

DOK Level Required by Task 

7.SP.6 




7.SP.7 




7.SP.8 





Appendix G presents additional considerations for use of these data as the Consortium moves forward 
with its test and item design and development. The suggestions presented in Appendix G are based on 
selected comments from members of the Consortium's Technical Advisory Committee. These 
suggestions are intended to generate and/or guide further discussion of this study's data among 
Consortium work groups and committees (e.g., Test Design, Item Development, Technology Approach, 
and Accessibility and Accommodations Work Groups, content specifications committee). 
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APPENDIX A: GUIDING PRINCIPLES AND CONSIDERATIONS 


SBAC Common Core State Standards Analysis 
Defining Eligible Content for the Summative Assessment 

Guiding Principles and Considerations (rev. 12/09/10) 

Below are guiding principles and considerations informing the analysis and determination of the 
Common Core State Standards (CCSS) content eligible for the Consortium's end-of-year summative 
assessment. The Consortium's Executive Committee and members of the Consortium's project 
management reviewed these guiding principles and considerations and provided input. 

Resources consulted: 

• CCSS standards, appendices, and explanatory documents 

• WestEd's SBAC paper, Research-Supported Guidance for the Development of a Comprehensive 
Assessment Framework Based on the Common Core State Standards, Sept. 30, 2010 

A. What CCSS content can/should be assessed? 

1. What is reasonable to assess on a summative assessment (i.e., students will have had the 
opportunity to learn the knowledge, skills, and abilities reflected in the standards)? 

• The CCSS were designed to address content for each grade level that is reasonable to 
learn in a school year, so the full set of standards (as described/specified in the CCSS 
documents) will be considered and analyzed for eligibility for the summative assessment. 

• ELA standards (Reading, Writing, Speaking & Listening, and Language strands) will be 
considered eligible for the Consortium's end-of-year summative assessment, but 
standards for Literacy in History/Social Studies, Science, and Technical Subjects will not be 
considered. (Note: Test designers may choose to use history/social studies, science, and 
technical subjects as context for assessing literacy skills; however, the individual standards 
for literacy in history/social studies, science, and technical subjects will not be included in 
this analysis— i.e., for eligible content for summative assessment 18 .) 

• The following will be considered for eligibility on the summative assessment: 

o ELA: Grades 9-10 and 11-12 

o Mathematics: All six conceptual categories (Note: Modeling is integrated into five 
of the conceptual categories) 

• Note: The high school assessment is not based on a single year of instruction; rather, it is 
cumulative across multiple years. 

• The specific content to be assessed must be determined by the test design and test and 
item specifications. Since the SBAC assessment intends to allow an item to assess multiple 
standards, for this analysis (i.e., of eligible content) analysts will tend toward being more 
inclusive than exclusive in determining which standards are eligible for the Consortium's 
summative assessment. 


18 The standards for Literacy in History/Social Studies, Science, and Technical Subjects overlap with those for ELA 
Reading and Writing in grades 6-12; the intent of these standards appears to be to draw awareness to and provide 
guidance for interdisciplinary teaching of literacy skills (see p. 4 of the Introduction to the CCSS for English 
Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects). 
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• The analysis of eligible content for the summative assessment will be limited to the CCSS 
and will not include/consider knowledge, skills, and abilities beyond those of the CCSS. 

2. What is the broadest level of granularity for ELA and mathematics that would be appropriate for 
assessment? For item specifications? 

• ELA: The main standard statements (numbered standards) have an appropriate level of 
content detail for assessment. The College and Career Readiness (CCR) Anchor Standards 
are too broad; the examples and many of the lettered "sub-standard" statements appear 
overly detailed for determining eligibility for summative assessment, but could be useful 
to consider in writing item specifications. As appropriate, the CCR Anchor Standard 
subheadings and statements will be considered in interpreting the standard and 
determining eligibility of the grade-specific standards. 

• Mathematics: The main standard statements (numbered standards) have an appropriate 
level of content detail for assessment. The standard cluster statement is generally too 
broad (and in some cases it is narrower than some of the numbered standards); the 
examples and many of the lettered "sub-standard" statements appear overly detailed for 
determining eligibility for summative assessment, but could be useful to consider in 
writing item specifications. 

As appropriate, the cluster statements will be considered in interpreting the standards 
and determining eligibility of the "numbered standards." 

• The analysis of eligible content for summative assessment focuses on a grain size that is 
appropriate for informing item and test specifications and development. The level of 
granularity at which the analysis is focused is intended to allow for flexibility in how 
performance on the test will be reported. The number of items needed and the 
narrowness/broadness of content coverage within an item are beyond the scope of this 
analysis and better suited for discussions of item and test specifications. 

3. How should we consider content emphasis? Is there any work/resources related to content 
progressions (or clusters) that reflect content priorities/emphases? 

• As appropriate, ELA CCR Anchor Standards that provide a set of content clusters and the 
introductions for each grade (i.e., the narrative describing the overall focus of the grade 
level) in the mathematics standards will be considered. 

• Some standards may need to be assessed within the context of assessing other standards 
(e.g., ELA Language standards may need to be assessed via a reading or writing task), 
given variation in grain size within the standards documents themselves and the relative 
importance of certain standards. 

For example, Grade 9-10 Language standard 1: "Demonstrate command of the 
conventions of standard English grammar and usage when writing or speaking." 

4. Is there anything/are there any considerations that would make content NOT eligible for 
summative assessment purposes? 

• Standards identified by the CCSS as not required for all students (e.g., the "(+)" standards 
in mathematics 

• Standards that are best measured by a portfolio of work or by tasks that must be 
completed over an extended period of time, that is, beyond the length of time allotted for 
performance tasks (e.g., ELA Reading standard 10 for all grades) 
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B. Structure of the analysis across content areas/grade levels 

1. Might there be a different product for grades 3-8 versus for high school? What 
similarities/differences might there be across grades within a content area (e.g., scope, number, 
structure)? 

• Generally, the protocol and rating tool/database will be the same for ELA and 
mathematics, across grades. 

2. How should we consider appropriate item types (e.g., selected response, constructed response, 
performance tasks)? 

• Items types are defined in the SBAC proposal— analysts will use these definitions to 
determine which item types are appropriate for assessing eligible content standards. 

• As mentioned previously (see #1), analysts will be more inclusive in order to provide the 
test designers with flexibility in their ultimate determination of which item type(s) to use 
for assessing eligible content. 

C. Process for determining eligibility for assessment 

1. Recommended approaches/considerations for ensuring accurate/appropriate interpretation of 
intent of standards/content meaning, understanding of introductory statements vis-a-vis 
individual related standards, etc.? 

• We will begin by relying on our analysts' expertise and crafting the criteria for analysis 
carefully. 

• We will rely on external review and comment by other experts (e.g., national experts in 
assessment, curriculum, instruction, standards, and the content areas). 

• We will review any relevant existing analyses. 

2a. What are recommended criteria for determining which standards are eligible for assessment? 

• The following general, widely accepted and validated criteria for assessability (for a 
summative tool) are proposed. These criteria also were considered vis-a-vis information 
presented in the SBAC proposal: 

o Measurable via on-demand tasks, oral or written response 
o Measurable via selected response, constructed response, and/or performance 
tasks (as defined in SBAC proposal) 
o Learnable within the school year (as based on CCSS documents) 
o Expected content for all students (based on CCSS documents) 

2b. What role does consideration of content on the vertical pathway to college and career 
preparation play in establishing criteria? 

• As appropriate, information in the CCSS about CCR Anchor Standards for ELA will be 
considered in interpreting standards and determining eligibility of the grade-specific 
standards. 

• Evaluating whether a given standard is on or appropriate for the pathway to college and 
career preparation is beyond the scope of this analysis (i.e., of eligible content). 
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2c. How should we account for possible achievement differentiators (e.g., considering growth) as 
we establish these criteria? 

• Ranges of cognitive complexity (DOK) of each standard determined to be eligible for the 
summative assessment will be indicated by the analysts. 

• Determination of actual/possible achievement differentiators is beyond the scope of this 
analysis (i.e., of eligible content). 

2d. How should we consider range of cognitive complexity as we establish these criteria? 

• Our draft criteria include rating each standard for the full range of relevant DOK levels. 

• The results of the DOK analysis of the standards can inform selection of standards to test a 
range and balance of cognitive complexity. 
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APPENDIX B. DESCRIPTIONS OF ITEM TYPES 


Selected response (multiple-choice) 

"Whereas selected-response items have routinely been a part of assessments of student learning, their 
emphasis has too often been on low-level skills reflecting knowledge of discrete pieces of information 
that are not critical for subsequent student learning. The emphasis here will be on the development of 
items that reflect important knowledge and skills consistent with the expectations of the CCSS. Items 
can be developed to address knowledge and skills from more than one standard. The appropriate and 
judicious use of selected-response items provides for a cost-effective means to address content in terms 
of test development, administration, and scoring." (SBAC, 2010b, p. 52) 

Extended constructed response 

"Our assessment design calls for the use of extended constructed-response items. These items will be 
used to assess knowledge and skills not easily assessed with selected-response or technology-enhanced 
items. Constructed-response items allow students to demonstrate their use of complex thinking skills 
such as formulating comparisons or contrasts; proposing cause and effects; identifying patterns or 
conflicting points of view; categorizing, summarizing, or interpreting information; and developing 
generalizations, explanations, justifications, or evidence-based conclusions (Darling-Hammond & 
Pecheone, 2010). These complex thinking skills are consistent with the expectations for college- and 
career-readiness and will be included in both the English language arts and mathematics assessments." 
(SBAC, 2010b, p. 53) 

Technology enhanced (multiple-choice or constructed response) and Technology enhanced constructed 
response (tied to writing performance event ) 19 

"The Consortium is committed to making effective use of technology by including items that capitalize 
on the capabilities of the assessment platform. The effective use of technology can expand not only the 
nature of the content that can be presented but also the knowledge, skills, and processes that can be 
assessed (Quellmalz & Moody, 2004). Technology-enhanced items can take advantage of drag-and-drop, 
hot spot, and simulation technologies along with the use of online tools to measure content that was 
previously not assessed or was assessed through constructed-response item formats requiring more 
elaborate scoring procedures. While most of the work with the technology-enhanced items has been 
done in the area of science, we envision the development of technology-enhanced items in both English 
language arts and mathematics. For example, a video presentation of a speech could be combined with 
a reading passage to provide students the opportunity to integrate reading and listening skills per the 
CCSS and evaluate content across diverse media." (SBAC, 2010b, pp. 52-53) 

Performance task 

"The summative evaluation for each student will also include performance events that will provide a 
measure of the student's ability to integrate knowledge and skills across multiple standards— a key 
component of college- and career-readiness. Performance events will be used to better measure 
capacities such as depth of understanding, research skills, and complex analysis, which cannot be 
adequately assessed with selected- or constructed-response items. At grades 3-8, students will engage 


19 The analysis for this study did not distinguish among these technology-enhanced item types; determining the 
appropriate technology-enhanced item type to best assess a standard or set of standards should accompany 
further work on clustering standards and developing test and item specifications and assessment task templates. 
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in two rigorous performance events for ELA and mathematics. At the high school level, students will 
engage in up to six performance events by grade 11 for both ELA and mathematics. These events will be 
computer-delivered and typically will require one to two class periods to complete." (SBAC, 2010b, 

P-42) 

"These events will evaluate the CCSS in ways that require more student-initiated planning, management 
of information and ideas, interaction with other materials and/or people, and production of more 
extended responses (e.g., oral presentations, exhibitions, product development, in addition to more 
extended written responses) that reveal additional abilities of students (Darling-Hammond & Pecheone, 
2010) not captured by the other item types included in the summative assessment." (SBAC, 2010b, p. 

53) 

Writing prompts 

For the purpose of this analysis, the following decision rule will be applied to define the writing prompt 
item type: 

"The item type 'writing prompt' is interpreted to refer to an on-demand task requiring the 
student to write for the purpose of assessing the students skills or knowledge in writing (that is, 
in the Writing and Language standards), and not for the purpose of assessing the student's skills 
or knowledge in reading (that is, the Reading standards)." 
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APPENDIX C. WEBB DEPTH OF KNOWLEDGE LEVELS 


The following descriptions of Webb's depth of knowledge (DOK) levels are excerpted from the Web 
Alignment Tool (WAT) Training Manual, Draft Version 1.1 (Webb, 2005, pp. 45-46 and 70-75). DOK 
levels for ELA and mathematics are described separately. 

ELA DOK Levels 

In language arts, four DOK levels were used to judge both reading and writing objectives and assessment 
tasks. The reading levels are based on Valencia and Wixson (2000, pp. 909-935). The writing levels were 
developed by Marsha Horton, Sharon O'Neal, and Phoebe Winter. 

Reading Level 1. Level 1 requires students to receive or recite facts or to use simple skills or abilities. 

Oral reading that does not include analysis of the text, as well as basic comprehension of a text, is 
included. Items require only a shallow understanding of the text presented and often consist of 
verbatim recall from text, slight paraphrasing of specific details from the text, or simple understanding 
of a single word or phrase. Some examples that represent, but do not constitute all of, Level 1 
performance are: 

• Support ideas by reference to verbatim or only slightly paraphrased details from the text. 

• Use a dictionary to find the meanings of words. 

• Recognize figurative language in a reading passage. 

Reading Level 2. Level 2 includes the engagement of some mental processing beyond recalling or 
reproducing a response; it requires both comprehension and subsequent processing of text or portions 
of text. Inter-sentence analysis of inference is required. Some important concepts are covered, but not 
in a complex way. Standards and items at this level may include words such as summarize, interpret, 
infer, classify, organize, collect, display, compare, and determine whether fact or opinion. Literal main 
ideas are stressed. A Level 2 assessment item may require students to apply skills and concepts that are 
covered in Level 1. However, items require closer understanding of text, possibly through the item's 
paraphrasing of both the question and the answer. Some examples that represent, but do not constitute 
all of, Level 2 performance are: 

• Use context cues to identify the meaning of unfamiliar words, phrases, and expressions that 
could otherwise have multiple meanings. 

• Predict a logical outcome based on information in a reading selection. 

• Identify and summarize the major events in a narrative. 

Reading Level 3. Deep knowledge becomes a greater focus at Level 3. Students are encouraged to go 
beyond the text; however, they are still required to show understanding of the ideas in the text. 

Students may be encouraged to explain, generalize, or connect ideas. Standards and items at Level 3 
involve reasoning and planning. Students must be able to support their thinking. Items may involve 
abstract theme identification, inference across an entire passage, or students' application of prior 
knowledge. Items may also involve more superficial connections between texts. Some examples that 
represent, but do not constitute all of, Level 3 performance are: 

• Explain or recognize how the author's purpose affects the interpretation of a reading selection. 

• Summarize information from multiple sources to address a specific topic. 

• Analyze and describe the characteristics of various types of literature. 
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Reading Level 4. Higher-order thinking is central and knowledge is deep at Level 4. The standard or 
assessment item at this level will probably be an extended activity, with extended time provided for 
completing it. The extended time period is not a distinguishing factor if the required work is only 
repetitive and does not require the application of significant conceptual understanding and higher-order 
thinking. Students take information from at least one passage of a text and are asked to apply this 
information to a new task. They may also be asked to develop hypotheses and perform complex 
analyses of the connections among texts. Some examples that represent, but do not constitute all of, 
Level 4 performance are: 

• Analyze and synthesize information from multiple sources. 

• Examine and explain alternative perspectives across a variety of sources. 

• Describe and illustrate how common themes are found across texts from different cultures. 

Writing Level 1. Level 1 requires the student to write or recite simple facts. The focus of this writing or 
recitation is not on complex synthesis or analysis, but on basic ideas. The students are asked to list ideas 
or words, as in a brainstorming activity, prior to written composition; are engaged in a simple spelling or 
vocabulary assessment; or are asked to write simple sentences. Students are expected to write, speak, 
and edit using the conventions of Standard English. This includes using appropriate grammar, 
punctuation, capitalization, and spelling. Students demonstrate a basic understanding and appropriate 
use of such reference materials as a dictionary, thesaurus, or Web site. Some examples that represent, 
but do not constitute all of, Level 1 performance are: 

• Use punctuation marks correctly. 

• Identify Standard English grammatical structures, including the correct use of verb tenses. 

Writing Level 2. Level 2 requires some mental processing. At this level, students are engaged in first- 
draft writing or brief extemporaneous speaking for a limited number of purposes and audiences. 
Students are expected to begin connecting ideas, using a simple organizational structure. For example, 
students may be engaged in note-taking, outlining, or simple summaries. Text may be limited to one 
paragraph. Some examples that represent, but do not constitute all of, Level 2 performance are: 

• Construct or edit compound or complex sentences, with attention to correct use of phrases and 
clauses. 

• Use simple organizational strategies to structure written work. 

• Write summaries that contain the main idea of the reading selection and pertinent details. 

Writing Level 3. Level 3 requires some higher-level mental processing. Students are engaged in 
developing compositions that include multiple paragraphs. These compositions may include complex 
sentence structure and may demonstrate some synthesis and analysis. Students show awareness of 
their audience and purpose through focus, organization, and the use of appropriate compositional 
elements. The use of appropriate compositional elements includes such things as addressing 
chronological order in a narrative, or including supporting facts and details in an informational report. At 
this stage, students are engaged in editing and revising to improve the quality of the composition. Some 
examples that represent, but do not constitute all of, Level 3 performance are: 

• Support ideas with details and examples. 

• Use voice appropriate to the purpose and audience. 

• Edit writing to produce a logical progression of ideas. 

Writing Level 4. Higher-level thinking is central to Level 4. The standard at this level is a multi-paragraph 
composition that demonstrates the ability to synthesize and analyze complex ideas or themes. There is 
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evidence of a deep awareness of purpose and audience. For example, informational papers include 
hypotheses and supporting evidence. Students are expected to create compositions that demonstrate a 
distinct voice and that stimulate the reader or listener to consider new perspectives on the addressed 
ideas and themes. An example that represents, but does not constitute all of, Level 4 performance is: 

• Write an analysis of two selections, identifying the common theme and generating a purpose 
that is appropriate for both. 

Mathematics DOK Levels 

Level 1 (Recall) includes the recall of information such as a fact, definition, term, or a simple procedure, 
as well as performing a simple algorithm or applying a formula. That is, in mathematics, a one-step, well 
defined, and straight algorithmic procedure should be included at this lowest level. Other key words that 
signify Level 1 include "identify," "recall," "recognize," "use," and "measure." Verbs such as "describe" 
and "explain" could be classified at different levels, depending on what is to be described and explained. 

Level 2 (Skill/Concept) includes the engagement of some mental processing beyond an habitual 
response. A Level 2 assessment item requires students to make some decisions as to how to approach 
the problem or activity, whereas Level 1 requires students to demonstrate a rote response, perform a 
well-known algorithm, follow a set procedure (like a recipe), or perform a clearly defined series of steps. 
Keywords that generally distinguish a Level 2 item include "classify," "organize," "estimate," "make 
observations," "collect and display data," and "compare data." These actions imply more than one step. 
For example, to compare data requires first identifying characteristics of objects or phenomena and 
then grouping or ordering the objects. Some action verbs, such as "explain," "describe," or "interpret," 
could be classified at different levels depending on the object of the action. For example, interpreting 
information from a simple graph, or reading information from the graph, also are at Level 2. Interpreting 
information from a complex graph that requires some decisions on what features of the graph need to 
be considered and how information from the graph can be aggregated is at Level 3. Level 2 activities are 
not limited only to number skills, but may involve visualization skills and probability skills. Other Level 2 
activities include noticing or describing non-trivial patterns, explaining the purpose and use of 
experimental procedures; carrying out experimental procedures; making observations and collecting 
data; classifying, organizing, and comparing data; and organizing and displaying data in tables, graphs, 
and charts. 

Level 3 (Strategic Thinking) requires reasoning, planning, using evidence, and a higher level of thinking 
than the previous two levels. In most instances, requiring students to explain their thinking is at Level 3. 
Activities that require students to make conjectures are also at this level. The cognitive demands at 
Level 3 are complex and abstract. The complexity does not result from the fact that there are multiple 
answers, a possibility for both Levels 1 and 2, but because the task requires more demanding reasoning. 
An activity, however, that has more than one possible answer and requires students to justify the 
response they give would most likely be at Level 3. 

Other Level 3 activities include drawing conclusions from observations; citing evidence and developing a 
logical argument for concepts; explaining phenomena in terms of concepts; and deciding which 
concepts to apply in order to solve a complex problem. 

Level 4 (Extended Thinking) requires complex reasoning, planning, developing, and thinking, most likely 
over an extended period of time. The extended time period is not a distinguishing factor if the required 
work is only repetitive and does not require applying significant conceptual understanding and higher- 
order thinking. For example, if a student has to take the water temperature from a river each day for a 
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month and then construct a graph, this would be classified as a Level 2. However, if the student is to 
conduct a river study that requires taking into consideration a number of variables, this would be a Level 
4. At Level 4, the cognitive demands of the task should be high and the work should be very complex. 
Students should be required to make several connections— relate ideas within the content area or 
among content areas— and have to select one approach among many alternatives on how the situation 
should be solved, in order to be at this highest level. Level 4 activities include designing and conducting 
experiments and projects; developing and proving conjectures, making connections between a finding 
and related concepts and phenomena; combining and synthesizing ideas into new concepts; and 
critiquing experimental designs. 
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APPENDIX D. LIST OF ANALYSTS' COMMENTS 


200 Series 

Expected 

201 

Standard is labeled with a (+), indicating that the content should be learned by students in order to 
take advanced courses. 

202 

A portion of this standard is labeled with a (+), indicating that the content should be learned by 
students in order to take advanced courses. 

300 Series 

Measurable 

301 

At least one verb in the standard requires the student to generate a response. 20 

302 

At least a portion of this standard is best measured by in-person observation. 21 

303 

Assessment of this standard (or a portion of this standard) may require guidance and support from 
an adult. 

304 

Assessment of this standard (or a portion of this standard) may require interaction with others. 

305 

This standard (or a portion of this standard) overlaps with one or more standards in another 
strand. 

306 

This standard (or a portion of this standard) overlaps with one or more other standards in this 
strand. 

307 

A portion of this standard is not measurable via on-demand assessment. 

308 

Full coverage of the standard may not be possible with selected response. 

309 

Statements in this standard inform the complexity of text and selection of content area texts 
appropriate for the assessment. 

310 

At least a portion of this standard is related to process rather than outcome. 2 

311 

This standard (or a portion of this standard) requires an extended amount of time. 

312 

Technological enhancement may be necessary for all item types for this standard. 23 

313 

The parameters for assessment may be difficult to define for a component of this standard. 

314 

Only listening would be measured if responses were written. 

315 

This standard may be used as a stand-alone for a performance task. 


20 Skills that require the student to generate a response are most effectively assessed via constructed-response 
items; however, a standard may describe multiple skills and thus be rated for multiple item types, not all of which 
apply to all of the content. 

21 The portion of the standard describing skills or knowledge best measured by in-person observations would not 
be considered eligible for the summative assessment. 

22 The portion of the standard describing process skills over extended time would not be considered eligible for the 
summative assessment. 

23 Technology-enhanced items can be selected response or constructed response; for some standards, analysts 
rated other applicable item types in addition to technology-enhanced items, and added this note as a 
consideration that, while the content could be assessed via various item types, effectively assessing the content 
may mean that technological enhancement will be required. 
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400 Series 


DOK 


401 Higher DOK applies only to a portion of the standard that is to be learned by students in order to 

take advanced courses. 
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APPENDIX E - SBAC Eligible Content Data Workbook: ELA 



English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

’CJ) 

LU 

o 

o 

04 

o 

o 

CO 

O 

o 

■'3" 

o 

o 

DOK Comment 
CODE 

RL.3.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.3.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.3.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.3.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.3.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.3.6 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.3.7 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.3.8 

NA 


NA 


NA 


NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 


RL.3.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.3.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


Rl.3.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.3.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.3.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.3.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.3.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


Rl.3.6 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.3.7 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.3.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

N 

N 


Rl.3.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

N 


Rl.3.1 0 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


RF.3.3 

Y 


Y 


Y 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

N 

N 

N 


RF.3.4 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.3.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.3.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.3.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.3.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

’CJ) 

LU 

o 

o 

04 

o 

□ 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

W.3.5 

Y 


Y 


Y 

303 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.3.6 

Y 


Y 


N 

303 

304 
310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.3.7 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


W.3.8 

Y 


Y 


Y 

307 

N 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


W.3.9 

NA 


NA 


NA 


NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

NA 


W.3.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.3.1 

Y 


Y 


N 

304 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.3.2 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

N 

Y 

Y 

Y 

N 

N 


SL.3.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

Y 

Y 

N 

N 


SL.3.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

Y 


SL.3.5 

Y 


Y 


Y 

307 

N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.3.6 

Y 


Y 


Y 

307 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.3.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.3.2 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.3.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


L.3.4 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.3.5 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.3.6 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.4.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.4.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.4.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

RL.4.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.4.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.4.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

N 


RL.4.7 

Y 


Y 


Y 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.4.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.4.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.4.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


Rl.4.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.4.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.4.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.4.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.4.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.4.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.4.7 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.4.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.4.9 

Y 


Y 


Y 

313 

N 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.4.1 0 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


RF.4.3 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

N 

N 

N 


RF.4.4 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.4.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.4.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.4.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.4.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.4.5 






303 

304 














Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.4.6 






303 

304 














Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.4.7 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

co 

1— 

CD 

O 

c 

CO 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

W.4.8 

Y 


Y 


Y 

307 

N 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


W.4.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


W.4.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.4.1 

Y 


Y 


N 

304 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 


SL.4.2 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

N 


SL.4.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

Y 

Y 

N 

N 


SL.4.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.4.5 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.4.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.4.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.4.2 

Y 


Y 


Y 

3OT 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.4.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


L.4.4 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.4.5 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.4.6 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.5.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.5.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.5.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.5.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.5.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.5.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.5.7 

Y 


Y 


Y 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.5.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

CD 

o 

o 

04 

o 

o 

CO 

O 

o 

■'3" 

o 

o 

DOK Comment 
CODE 

RL.5.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


RL.5.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


Rl.5.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.5.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.5.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.5.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.5.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.5.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.5.7 

Y 


Y 


Y 

313 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.5.8 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.5.9 

Y 


Y 


Y 

313 

N 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.5.1 0 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


RF.5.3 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

N 

N 

N 


RF.5.4 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.5.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.5.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.5.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.5.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.5.5 






301 

303 



















304 














Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.5.6 






303 

304 














Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.5.7 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 


W.5.8 

Y 


Y 


Y 

307 

N 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


W.5.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


W.5.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 


SL.5.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.5.2 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

N 


SL.5.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

N 


SL.5.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.5.5 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.5.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.5.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.5.2 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.5.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


L.5.4 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.5.5 

Y 


Y 


Y 

305 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.5.6 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.6.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.6.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.6.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.6.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.6.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.6.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.6.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.6.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.6.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.6.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


Rl.6.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.6.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

Rl.6.3 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.6.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.6.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.6.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.6.7 

Y 


Y 


Y 

312 

313 

N 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.6.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.6.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.6.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.6.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.6.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.6.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.6.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.6.5 

Y 


Y 


Y 

301 

303 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.6.6 

Y 


Y 


N 

304 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.6.7 

Y 


Y 


Y 

307 

311 

313 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.6.8 

Y 


Y 


Y 

301 

308 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 


W.6.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W.6.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.6.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 


SL.6.2 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.6.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

ED 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.6.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.6.5 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.6.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.6.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.6.2 

Y 


Y 


Y 

3U4 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.6.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


L.6.4 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


L.6.5 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.6.6 

Y 


Y 


Y 

305 

307 

313 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.7.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.7.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.7.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.7.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.7.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.7.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.7.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.7.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.7.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.7.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


Rl.7.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

’CJ) 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

0 

o 

o 

DOK Comment 
CODE 

Rl.7.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.7.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.7.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.7.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.7.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.7.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.7.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.7.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.7.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.7.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.7.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.7.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.7.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.7.5 

Y 


Y 


Y 

301 

303 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.7.6 

Y 


Y 


N 

304 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.7.7 

Y 


Y 


Y 

307 

311 

313 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W.7.8 

Y 


Y 


Y 

301 

308 

313 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 


W.7.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W.7.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.7.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.7.2 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.7.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.7.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.7.5 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.7.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


1 7.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.7.2 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.7.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


L.7.4 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


L.7.5 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.7.6 

Y 


Y 


Y 

305 

307 

313 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.8.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.8.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.8.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.8.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.8.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.8.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.8.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.8.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.8.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.8.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge [ 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

0 

04 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

Rl.8.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.8.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.8.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.8.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl.8.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.8.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl.8.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.8.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl.8.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl.8.1 0 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.8.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.8.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.8.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.8.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W.8.5 

Y 


Y 


Y 

301 

303 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W.8.6 

Y 


Y 


N 

304 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W.8.7 

Y 


Y 


Y 

307 

311 

313 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W.8.8 

Y 


Y 


Y 

301 

313 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 


W.8.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W.8.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.8.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.8.2 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.8.3 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.8.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.8.5 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.8.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.8.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.8.2 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.8.3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


L.8.4 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


L.8.5 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


L.8.6 

Y 


Y 


Y 

305 

307 

313 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL. 9-10.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.9-10.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.9-10.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RL.9-10.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.9-10.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.9-10.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.9-10.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.9-10.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.9-10.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL. 9-10. 10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge [ 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

0 

04 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

Rl. 9-10.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RI.9-1 0.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl .9- 1 0.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RI.9-1 0.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RI.9-1 0.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


RI.9-1 0.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RI.9-1 0.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RI.9-1 0.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RI.9-1 0.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RI.9-1 0.1 0 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W. 9-10.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 9-10. 2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 9-10. 3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 9-10. 4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 9-10. 5 

Y 


Y 


Y 

301 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W. 9-10. 6 

Y 


Y 


N 

304 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W. 9-10. 7 

Y 


Y 


Y 

307 

311 

313 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W. 9-10. 8 

Y 


Y 


Y 

301 

313 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 


W. 9-10. 9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W. 9-10. 10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.9-10.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 


SL.9-10.2 

Y 


Y 


Y 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.9-10.3 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.9-10.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.9-10.5 

Y 


Y 


Y 

313 

N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.9-10.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L. 9-10.1 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L. 9-10. 2 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L. 9-10. 3 

Y 


Y 


Y 

301 

305 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


L. 9-1 0.4 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


L. 9-10. 5 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


L. 9-10. 6 

Y 


Y 


Y 

305 

307 

313 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


RL.1 1-12.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.1 1-12.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.1 1-12.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


RL.1 1-12.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


RL.1 1-12.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.1 1-12.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.1 1-12.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.1 1-12.8 

N/A 


N/A 


N/A 


N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


RL.1 1-12.9 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


RL.1 1-12.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


Rl. 11-12.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge | 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

’CJ) 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

0 

o 

o 

DOK Comment 
CODE 

Rl. 11-12.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl. 11-12.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl. 11-12.4 

Y 


Y 


Y 

305 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 


Rl. 11-12.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl. 11-12.6 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl. 11-12.7 

Y 


Y 


Y 

308 

312 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

Y 

Y 


Rl. 11-12.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

Y 


Rl. 11-12.9 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


Rl. 11-12.10 

Y 


Y 


N 

309 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W. 11-12.1 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 11-12.2 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 11-12.3 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 11-12.4 

Y 


Y 


Y 

306 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

Y 


W. 11-12.5 

Y 


Y 


Y 

301 

304 

305 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 


W. 11-12.6 

Y 


Y 


N 

304 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

N 

N 


W. 11-12.7 

Y 


Y 


Y 

307 

311 

313 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W. 11-12.8 

Y 


Y 


Y 

301 

313 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 


W. 11-12.9 

Y 


Y 


Y 

305 

N 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


W. 11-12.10 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 


SL.1 1-12.1 

Y 


Y 


N 

304 

311 

N 

N 

N 

N 

N 

N 

N 

Y 

Y 

Y 

N 


SL.1 1-12.2 

Y 


Y 


Y 

313 

314 

N 

N 

Y 

N 

N 

N 

Y 

N 

N 

Y 

Y 
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English Language Arts 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge j 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/3 

03 

1— 

CD 

O 

c 

CD 

E 

CD 

CL 

Writing Prompt 
(ELA ONLY) 

Oral Response 
Required 

CD 

-Q 

'or 

LU 

o 

o 

CNJ 

o 

o 

CO 

O 

o 

■'3- 

o 

o 

DOK Comment 
CODE 

SL.1 1-12.3 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

N 

Y 

N 

Y 

Y 

Y 


SL.1 1-12.4 

Y 


Y 


Y 


N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

Y 

N 


SL.1 1-12.5 

Y 


Y 


Y 

313 

N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

Y 

N 


SL.1 1-12.6 

Y 


Y 


Y 

305 

N 

N 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

N 


L.1 1-12.1 






301 

305 














Y 


Y 


Y 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


L.11-12.2 






301 

305 














Y 


Y 


Y 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


L.1 1-12.3 






301 

305 














Y 


Y 


Y 

308 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


L.1 1 -1 2.4 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 

N 


L.1 1-12.5 

Y 


Y 


Y 

305 

307 

Y 

N 

Y 

Y 

N 

N 

Y 

N 

Y 

Y 

N 


L.1 1-12.6 






305 

307 














Y 


Y 


Y 

313 

Y 

N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

CO 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

3.0A.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.0A.8 

Y 


Y 


Y 

302 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


3.0A.9 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


3.NBT.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


3.NBT.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.NBT.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.NF.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.NF.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.NF.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


3.MD.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.4 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


3.MD.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.MD.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Mathematics 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

CD 

_Q 

'or 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

3.G.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


3.G.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.0A.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.0A.2 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.0A.3 

Y 


Y 


Y 

301 

302 
308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


4.0A.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.0A.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.NBT.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NBT.2 

Y 


Y 


Y 

301 

302 
308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NBT.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NBT.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NBT.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.NBT.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.NF.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


4.NF.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


4.NF.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


4.NF.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.NF.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NF.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.NF.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


4.MD.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 
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Mathematics 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

CD 

_Q 

'or 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

4.MD.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.MD.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.MD.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.MD.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.MD.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.MD.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.G.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


4.G.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


4.G.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.0A.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.0A.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.0A.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NBT.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.NBT.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NBT.3 

Y 


Y 


Y 

301 
308 

302 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.NBT.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.NBT.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.NBT.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NBT.7 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


5.NF.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/5 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

5.NF.2 

Y 


Y 


Y 

302 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


5.NF.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NF.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NF.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


5.NF.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.NF.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.MD.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.MD.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.MD.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.MD.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.MD.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.G.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


5.G.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.G.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


5.G.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.RP.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.RP.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.RP.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.NS.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. NS. 2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. NS. 3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. NS. 4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. NS. 5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. NS. 6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 
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Mathematics | 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

CD 

_Q 

'or 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

6. NS. 7 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. NS. 8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.EE.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. EE. 2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. EE. 3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. EE. 4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. EE. 5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6. EE. 6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. EE. 7 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. EE. 8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6. EE. 9 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


6.G.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.G.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.G.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.G.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.SP.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6.SP.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.SP.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


6.SP.4 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


6.SP.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


7.RP.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

CO 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

’O) 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

7.RP.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.RP.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.NS.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7. NS. 2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7. NS. 3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.EE.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


7. EE. 2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7. EE. 3 

Y 


Y 


Y 

301 

302 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


7. EE. 4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


7.G.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.G.2 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.G.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.G.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.G.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.G.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


7.SP.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


7.SP.2 

Y 


Y 


Y 

301 

308 

315 

Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


7.SP.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


7.SP.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


7.SP.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 
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Mathematics | 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

CD 

_Q 

'or 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

7.SP.6 

Y 


Y 


Y 

308 

315 

Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


7.SP.7 

Y 


Y 


Y 

301 

308 

315 

Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


7.SP.8 

Y 


Y 


Y 

301 

308 

315 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8.NS.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


8. NS. 2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.EE.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


8. EE. 2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


8. EE. 3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8. EE. 4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8. EE. 5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8. EE. 6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8. EE. 7 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8. EE. 8 

Y 


Y 


Y 

301 

308 

310 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8.F.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.F.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.F.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.F.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8.F.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

CO 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

8.G.1 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

N 

N 


8.G.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.G.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.G.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.G.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8.G.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


8.G.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.G.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.G.9 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.SP.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


8.SP.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.SP.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


8.SP.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


N.RN.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


N.RN.2 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


N.RN.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


N.Q.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


N.Q.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


N.Q.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


N.CN.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


N.CN.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


N.CN.3 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.CN.4 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.CN.5 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/5 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

o 

o 

DOK Comment 
CODE 

N.CN.6 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.CN.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


N.CN.8 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.CN.9 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.1 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.2 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.3 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.4 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.5 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.6 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.7 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.8 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.9 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.1 0 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


N.VM.1 1 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


N.VM.1 2 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


A.SSE.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.SSE.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.SSE.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.SSE.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A.APR.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


A. APR. 2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A. APR. 3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

CD 

_Q 

'or 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

A. APR. 4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A. APR. 5 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


A. APR. 6 

Y 


Y 


Y 

301 

308 

310 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A. APR. 7 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


A.CED.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.CED.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.CED.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A.CED.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


A.REI.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A.REI.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.REI.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


A.REI.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A.REI.5 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


A.REI.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.REI.7 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


A.REI.8 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

N 

N 

N 


A.REI.9 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


A.REI.1 0 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


A.REI.1 1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


A.REI.1 2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

CO 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

o 

o 

DOK Comment 
CODE 

F.IF.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


F.IF.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


F.IF.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.7 

Y 


Y 

202 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.IF.9 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.BF.1 

Y 


Y 

202 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.BF.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.BF.3 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.BF.4 

Y 


Y 

202 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.BF.5 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


F.LE.1 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


F.LE.2 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.LE.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.LE.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


F.LE.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.TF.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


F.TF.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.TF.3 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 



WestEd 


Mathematics-1 1 


March 4, 2011 





APPENDIX E - SBAC Eligible Content Data Workbook: Mathematics 
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Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/5 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

LU 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

F.TF.4 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

N 

Y 

N 

N 


F.TF.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


F.TF.6 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


F.TF.7 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


F.TF.8 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


F.TF.9 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


G.C0.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


G.C0.2 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.C0.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.C0.4 

Y 


Y 


N 

310 

N 

N 

N 

N 

N 

N 

N 

Y 

N 

N 


G.C0.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.C0.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.C0.7 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


G.C0.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


G.C0.9 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 


G.CO.IO 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 


G.C0.11 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 


G.C0.12 

Y 


Y 


Y 

302 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


G.C0.13 

Y 


Y 


Y 

302 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


G.SRT.1 

Y 


Y 


N 

302 

N 

N 

N 

N 

N 

N 

N 

Y 

N 

N 


G.SRT.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.SRT.3 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


G.SRT.4 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

Performance Task 

Oral Response 
Required 

Eligible 

DOK 1 

DOK 2 

DOK 3 

DOK 4 

DOK Comment 
CODE 

G.SRT.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


G.SRT.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.SRT.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.SRT.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.SRT.9 

Y 


N 

201 

Y 


N 

Y 

Y 

Y 

N 

N 

N 

Y 

Y 

N 


G.SRT.10 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


G.SRT.1 1 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


G.C.1 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 


G.C.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.C.3 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


G.C.4 

Y 


N 

201 

Y 


N 

Y 

Y 

Y 

N 

N 

N 

Y 

N 

N 


G.C.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


G.GPE.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


G.GPE.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.GPE.3 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


G.GPE.4 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

N 

Y 

N 


G.GPE.5 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


G.GPE.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.GPE.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.GMD.1 

Y 


Y 


Y 


N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


G.GMD.2 

Y 


N 

201 

Y 


N 

Y 

Y 

Y 

N 

N 

N 

Y 

Y 

N 


G.GMD.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.GMD.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.MG.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


G.MG.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

C/5 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

CD 

_Q 

'cp 

CD 

o 

o 

CM 

o 

o 

CO 

O 

o 

^J- 

o 

o 

DOK Comment 
CODE 

G.MG.3 

Y 


Y 


Y 

315 

Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

Y 


S.ID.1 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.5 

Y 


Y 


Y 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.6 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.8 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.ID.9 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.IC.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


S.IC.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.IC.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.IC.4 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

N 

N 


S.IC.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


S.IC.6 

Y 


Y 


Y 

315 

N 

Y 

Y 

Y 

N 

Y 

N 

Y 

Y 

N 


S.CP.1 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.CP.2 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

N 

N 

N 


S.CP.3 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.CP.4 

Y 


Y 


Y 

301 

308 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.CP.5 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Y 

N 


S.CP.6 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.CP.7 

Y 


Y 


Y 


Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

N 

N 


S.CP.8 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 
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Mathematics ] 


Learnable 

Expected 

Measurable 

Item Type 

Eligible 

Depth of Knowledge ] 


YorN 

CODE 

YorN 

CODE 

YorN 

CODE 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

YorN 

CODE 

CCS Code 

Learnable during 
school year 

Learnable 
Comment CODE 

Expected of all 
Students 

Expected 
Comment CODE 

Measurable via on- 
demand 

Measurable 
Comment CODE 

Selected 

Response 

Extended 

Constructed 

Response 

Technology 

Enhanced 

CO 

03 

1— 

CD 

O 

£= 

03 

E 

CD 

CL 

Oral Response 
Required 

Eligible 

o 

o 

CM 

o 

o 

CO 

O 

o 

o 

Q 

DOK Comment 
CODE 

S.CP.9 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


S.MD.1 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


S.MD.2 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


S.MD.3 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


S.MD.4 

Y 


N 

201 

Y 

301 

308 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


S.MD.5 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

Y 

N 


S.MD.6 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

N 


S.MD.7 

Y 


N 

201 

Y 


Y 

Y 

Y 

Y 

N 

N 

N 

Y 

Y 

N 
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APPENDIX F. PRELIMINARY REPORT REVIEW— FEEDBACK AND RESPONSES 


SMARTER Balanced Assessment Consortium Common Core State Standards Analysis: Defining Eligible 

Content for the Summative Assessment 

All Consortium member states and work groups were provided the opportunity to review and comment 
on a draft version of this report (Feb. 4, 2011 version). The purpose of the review was to solicit comment 
related to the clarity of the report from States. Three work groups were identified as the primary 
audience for this report: Test Design, Item Development, and Technology Approach. From these three 
work groups, feedback was solicited regarding the clarity of the report, as well as the usefulness and 
meaningfulness of the information presented in the report vis-a-vis the work groups' summative 
assessment-related tasks. 

Below are the comments received from reviewers. First are comments from reviewers describing 
information that they found clear/useful. Table F-l lists comments that (a) are addressed in this final 
report and (b) are beyond the scope of this project and require follow-up by work groups, the Executive 
Committee, and/or the Consortium's Technical Advisory Committee (TAC).* The comment source is 
provided, along with the response/action to each comment, and when comments require responses by 
work groups, the Executive Committee, or the TAC, a recommended follow-up by one or more of these 
groups is specified. 

* Some comments have been addressed in the report, as noted in Table F-l. Comments that require responses 
beyond the scope of this project/report are included, and recommendations for follow-up with Consortium work 
groups/committees, the Executive Committee, and/or the TAC are noted, in Table F-l. Cross-work 
group/committee communication and coordination are encouraged as noted. 

Comments: Clear/Useful: 

1. The limitations pointed out at the bottom of page 1 are helpful. 

2. I think Table 27 is a great way to highlight implications for the various work groups. The 
examples on pages 35 and 36 are quite useful. 

3. Especially useful were the statements of what the purposes of the document were NOT 
intended to be. 

4. what a great report - this is a home run in a draft form 

5. Pg. 35 - I like the ideas presented here 

6. I looked at the draft through the lens of ELA. The purpose of the study was explicitly clear. The 
details and methodology were explained in straightforward, precise terms that are easily 
understood. Anyone with knowledge of the contents of the common core should be able to 
understand the methodology and interpret it to stakeholders. I was impressed by both the level 
of specificity and the attention paid to addressing the key questions. Key findings were 
described and key terms were paraphrased and defined as needed. Overall, I found that the 
document shed light on the work of the SBAC as the consortium determines eligible content for 
assessments. 

7. Purpose: Clear - very useful for defining eligible content for summative assessment. Our 
impression is that the main purpose behind this project was to look at each of the Common Core 
State Standards and to analyze if they should be included in a Summative Assessment for the 
assigned grade level, and how the standard would be best assessed based on item type. This 
would be very useful knowledge for development of a Test Specifications document, and it does 
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give some hints as to what item type would be most appropriate for assessing each of the 
standards. The body of the document provides specific analysis as to which of the general ELA 
standards would be assessed and how. The appendix provides the tables showing that same 
information for the specific grade level standards. 

8. Overall findings are clear and are useful to give a general sense of the big picture, but the 
specific implications/uses were even more helpful. 

9. Overall this document clarifies how the review of the standards was completed and the process 
for how decision were reached about which standards would be assessed and how. 

10. Intent and Content of Report 

a. The report fulfills its intent of addressing the range of eligible content for a summative 
assessment system. The next steps for workgroups will require some heavy lifting as far 
as detailed designs by standards, but this sets a basis for that work and offers some 
warranted suggestions (pgs. 35 and 36) for accomplishing this. 

b. For the most part, I believe the content is accurate. I think the use of Technology 
Enhanced items and Performance Tasks were slightly "undersold", but not significantly. 

11. The purpose is clearly described with appropriate references to the original application and 
deliverables statements 

12. In general, the presentation of information was clear, although it did take some concentrated 
reading. 

a. From mathematics perspective: the document is organized and explained well, so we 
could use it if working to design test or item specifications. 

b. From the ELA perspective: the document is well-organized and easy to follow. There are 
several comment codes that are referred to in various tables in the document, but a 
quick check of the body of the document and the appendices clarified what these codes 
were reflecting. One of those comments was "the parameters for assessment may be 
difficult to define for a component of this standard," which could use some clarification. 

13. The section detailing the reviewer's definitions of various coding, such as that used to 
distinguish verbs in the various standards, is very helpful, and should inform future committee 
work as well as test blueprint and item design in RFP. 

14. Overall, I am very pleased to see the focus on multiple types of items. 

15. The purpose, key questions, and history sections help clarify the purpose of the study. 

16. We appreciate the clarification (under Decision Rules) of the general verbs used in the CC, as 
well as the examples for mathematics and ELA. The decision to use authored text is contrary to 
our current assessment criteria for reading, but we are glad that was made clear in this 
document at this stage of development. It represents a very thorough analysis of every standard 
in terms of its eligibility on a number of dimensions. With regard to the clarity of the report, I 
find the content very well-organized, lucid and accessible. Overall, I am highly impressed with 
the thoroughness of these findings and believe that they will provide a solid framework from 
which assessment items will be developed. 

17. The document provides strong guidance for development of formative tools and assessment, 
particularly with regard to standards that cannot be assessed at full coverage. The Charts with 
codes clearly connect to the learning targets within specific standards. We believe the 
considerations and implications are clearly articulated. We really believe it is an impressive draft 
and find it quite useful to our work. 

18. Purpose of the study was clear. The context and information needed to move into the more 
complex job of defining the eligible content was provided. 

19. Math findings are particularly clear 

20. The terminology was clear and the report was easily read. 
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21. Very well-done. It was helpful to have the process outlined in a step-by-step manner. 

22. Nice to hear the training was only completed when the analysts had mastered the entire 
process. 

23. Once the process and criteria were defined, the findings were clear. 

24. The findings and the way they are reported will support the people working with them. It was 
helpful that the process for how to assess multiple standards by the use of a performance task 
was explained. 

25. It is good to emphasis that this is not an analysis of the quality of the standards as that could 
take the group in an entirely different direction. 


Table F-l. Responses to Feedback and Recommended Follow-Up 






Recommended 





Follow-Up 


Reviewer Comment 

Comment Source 

Action/Response from 
Report's Authors 

Work Group(s) 

Executive 

Committee 

u 

< 

H 

1 . 

pp 3-4 - Table 2 - please 

Tech. WG 

This has been addressed in 





explain letters in parenthesis 
in the Grade Standard 
descriptions for Writing and 
Language 


the final report 




2. 

pg 6 - Population - if 

Tech. WG 

a. Response to this 

X 




summative assessment is 


question is beyond the 





intended for all students, 


scope of this 





including students from 


project/report. 





special populations: 


Suggest follow up by 





a. Will items be maintained 


consortium's work 





in a separate item 


groups: test design; item 





selection pool? 


development; technology 
approach; accessibility 
and accommodations. 





b. Will unique items be 


b. Response to this 

X 




written for the special 


question is beyond the 





population or will general 


scope of this 





assessment items be 


project/report. 





modified for presentation 


Suggest follow up by 





(including fewer 


consortium's work 





options)? 


groups: test design; item 
development; 
accessibility and 
accommodations. 





c. Will lower DOK levels be 


c. Response to this question 

X 




extended to provide 


is beyond the scope of 





differentiation of 


this project/report. 





cognitive skills measured 


Suggest follow up by 
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Reviewer Comment 

Comment Source 

Action/Response from 
Report's Authors 

Recommended 

Follow-Up 

Work Group(s) 

Executive 

Committee 

u 

< 

H 


for cognitively-impaired 
students? 


consortium's work 
groups: test design; item 
development; 
accessibility and 
accommodations. 




3. 

Pg 6 - Item Types - will 
extended constructed 
response items be scored 
against a rubric, multiple 
rubrics, weighted rubrics, 
standard or custom (item- 
specific) rubrics? 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 
work groups: test design; 
item development. 

X 



4. 

Pg 6 - Test Format - 
computer adaptive 
assessment - what criteria 
(attributes) will be the basis 
for item selection? 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 
work groups: test design; 
item development; 
technology approach. 

X 



5. 

Pg 8 - Measurable via On- 
Demand tasks - could a 
delivery module that 
includes a test administrator 
using a scoring rubric be 
incorporated to include 
tasks that require in-person 
observation or interaction? 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
additional discussion 
regarding summative 
assessment item types and 
follow up by consortium's 
work groups: test design; 
item development; 
administration. (Related to 
10b below) 

X 



6. 

Pg 8 - Item Type - ...to be 
completed over a number of 
days... - what are acceptable 
parameters that the 
technology work group 
needs to consider regarding 
student re-entry into an 
extended response or 
performance task item? 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 
work groups: item 
development; accessibility 
and accommodations; 
administration. (Related to 
lOe below) 

X 



7. 

Pg 9 - will the system need 
to capture oral responses by 
the student? If yes, how will 
responses be scored? If no, 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 

X 
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will these items be 
administered and scored 
with a rubric by the test 
administrator? 


work groups: test design; 
item development; 
accessibility and 
accommodations; 
technology approach, 
administration. 




8. 

Pg 11 - please clarify the 
distinction among gridded 
response, short-answer, 
open-ended, open- 
response, and constructed 
response items 

Tech. WG 

Response to this question is 
beyond the scope of this 
project/report. 

Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). (Related to 
25, 39, 72, 88, 89, 128, and 
134) 

X 



9. 

p.21 - 6. Use technology, 
including the Internet, to 
produce and publish writing 
and to interact and 
collaborate with others. 

a. Standard W.x.6 is not 
eligible for assessment 
via the proposed item 
types as this standard 
does not describe 
assessable content per 
se but instead "is 
related to process 
rather than outcome" 
(code 310). 

b. While this probably 
does not lend itself to 
an on-demand 
summative 
assessment, it should 
still be considered for 
formative assessment 
or as a part of a 
performance 
assessment, and not 

Tech. WG 

The focus of this study is on 
eligible content for the 
consortium's summative 
assessment. 

Content considerations for 
formative assessment 
should be undertaken by the 
Formative Processes and 
Tools/Professional 
Development Work group. 

X 
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simply dismissed as 
not accessible. 






10. 

Comment codes: 

Tech. WG 






a. 301 - does this imply 


a. This has been addressed 





that the item type must 
be extended response? 


in the final report 





b. 303-304 see question 5 


b. Response to this question 

X 




(i.e., Measurable via On- 


is beyond the scope of this 





Demand tasks - could a 


project/report. Suggest 





delivery module that 


additional discussion 





includes a test 


regarding summative 





administrator using a 


assessment item types 





scoring rubric be 


and follow up by 





incorporated to include 


consortium's work groups: 





tasks that require in- 


test design; item 





person observation or 


development; 





interaction?) 


administration. (Related to 
#5 above) 





c. Does the system need 


c. Response to this question 

X 




to ensure that all 


is beyond the scope of this 





standards are assessed? 


project/report. Suggest 





What are the minimum 


follow up by consortium's 





and maximum 


work groups: test design; 





parameters for 


item development; 





establishing that a 
standard has been 
assessed? 


reporting. 





d. 310 - please confirm - 


d. This has been addressed 





standards with code 310 
are not assessable on 
the summative 
assessment 


in the final report 





e. 311 - are these 


e. Response to this question 

X 




assessable, but need 


is beyond the scope of this 





consideration for 


project/report. Suggest 





student re-entry (see # 


follow up by consortium's 





6- Item Type - ...to be 


work groups: item 





completed over a 


development; accessibility 





number of days... - what 


and accommodations; 





are acceptable 


administration. (Related to 





parameters that the 
technology work group 
needs to consider 
regarding student 


#6 above) 
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re-entry into an 
extended response or 
performance task item?) 
into the assessment or 
are standards coded 311 
not assessable on the 
summative assessment? 







f. 312 - please clarify this 


f. This has been addressed in 





comment code - it 
states that technology 
enhancement may be 
required for all item 
types, yet Technology 
Enhanced is an item 
type. Please make the 
distinction and provide 
examples of both a 
technology enhanced 
item type, and another 
item type that is 
technologically 
enhanced 


the final report 





g. 315 - please provide 


g. This has been addressed 

X 




additional information 


in the final report. 





or an example of how 
this would be included 


Follow up by the 





in a summative 


Performance Tasks work 





assessment (page 43) 


group also is suggested. 




11. 

It may be premature at this 

State 1 

Response to this question is 

X 




point, but the descriptions 


beyond the scope of this 





of item types in Appendix B 


project/report. 





didn't answer a few 


The authors of this report 





questions 1 had when 


anticipate that there will be 





reading some of the 


further 





summary tables and the 


refinement/definition of 





tables in Appendix E 


each item type (e.g., work of 





(particularly with respect to 


various work groups such as 





the capabilities of 

technology-enhanced 

items). 


item development). 





a. For example, in Table 7, 


a. We encourage discussion 

X 




all 285 ELA standards were 


of points such as the one 





measurable via T-E items, 


you raise. The coding of 
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but performance tasks 
were more limited. 1 was 
wondering if SL 3.2 and SL 
3.3 wouldn't also be 
measurable in the context 
of performance tasks. 

b. Perhaps in a later version 
of the document, the 
analysts could comment 
(or code) the specific 
means by which the 
standard could be 
measured. (Although this 
may be the responsibility 
of test design work 
groups.) 


the standards is intended 
to inform discussion. 
Suggest follow up by 
consortium's work groups 
(e.g., test design, item 
development, 
performance tasks, 
technology approach, 
administration) 

b. Because we expect that 
the description/definition 
of each item type will 
undergo refinement (e.g., 
as work groups undertake 
their various 
projects/activities), we 
refrained from evaluating 
at this time which item 
type would best measure 
a given standard. Follow 
up by item development 
and test design work 
groups is recommended. 

X 



12. 

1 was curious about other 
protocols that might get at 
the degree to which a 
standard is measurable 
using each item type. Did 
the analysts feel the Yes/No 
approach limiting? To some 
extent, the comment codes 
get around this. 

State 1 

Because we expect that the 
description/definition of 
each item type will undergo 
refinement (e.g., as work 
groups undertake their 
various projects/activities), 
we refrained evaluating the 
degree of appropriateness 
of each item type for 
measuring a given standard. 
Suggest follow up by 
consortium's work groups 
(e.g., item development, 
performance tasks, 
technology approach, 
administration, accessibility 
and accommodations) 

X 



13. 

With regard to the findings, 1 
have one question: Will 
other experts be invited to 

Formative Processes and 
Tools/Professional DevWG 

The findings will be/have 
been reviewed by the 
consortium state members, 

X 

X 

X 
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perform a similar study and 
offer their findings? 1 see 
that a review will occur but 
it is not clear what exactly 
that will entail. 


including the consortium 
work groups, Executive 
Committee members, and 
members of the 
consortium's Technical 
Advisory Committee. 

It is recommended that 
interpretation of the 
findings in terms of 
implementation (e.g., test 
design, item specifications, 
content clustering) is 
reviewed by experts (e.g., 
external experts, the TAC, 
PARCC representatives) as 
relevant work 

groups/committees use this 
study's findings to inform 
their 

design/development/imple 
mentation activities 




14. 

On page 11 under the 
heading ELA Only, #1. 1 find 
this statement to be 
confusing. "The term "text" 
is interpreted to refer to an 
intact piece of authored 
writing..." Is this reference 
the use of the term text in 
the CCSS? If so, it seems 
much more limited that that 
intended by the CCSS. 
Clarification is needed here. 

Formative Processes and 
Tools/Professional DevWG 

This has been addressed in 
the final report 




15. 

On page 38 under What 
CCSS content can/should be 
assessed? Bullet 2 indicates 
that the literacy standards 
will not be considered 
eligible. More clarification is 
needed on this decision if 
possible. We are receiving a 
great many mixed messages 
with regard to this issue. 

Formative Processes and 
Tools/Professional DevWG 

This has been addressed in 
the final report 
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16. 

On page 2 paragraph 2, 
"each strand is headed by a 
strand- a specific set of 
College and Career 
Readiness (CCR) Anchor 
Standards that is identical 
across all grade levels"- 1 
don't understand what is 
meant by each strand is 
headed by a strand. 

Formative Processes and 
Tools/Professional DevWG 

This correction has been 
made in the report (...a 
strand-specific set...) 




17. 

What grade level are we 
assessing for ELA? It would 
seem we would have to do 
grade 10 or grade 12 
because the way the 
standards are organized by 
grade bands. That is if we 
did just grade 11, the 
expectation doesn't seem 
clear in the core that all the 
stuff in the grade 11-12 
band needs to be mastered 
by grade 11. However 1 can 
not tell from the report how 
this is being handled. 

Formative Processes and 
Tools/Professional DevWG 

Response to this issue is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work group: 
test design. 

X 



18. 

It is not clear what decision 
has been made about the 
content of the high school 
mathematics assessment. 1 
understand that one of the 
main purposes of the report 
is to identify which 
standards are able to be 
assessed and in which 
formats. However, there 
appears to be an implication 
that all of the high school 
mathematics standards that 
have been deemed as 
"assessable" must be 
learned by the end of the 
11 th grade. In the Common 
Core State Standards 
document, all of the "non- 

Formative Processes and 
Tools/Professional DevWG 

The purpose of this 
study/report is to evaluate 
the eligibility of the content 
of the CCSS for the 
consortium's summative 
assessment. The project's 
analysts were tasked with 
analyzing all standards in 
grades 3-8 and high school 
for ELA and mathematics 
(grades 9-10 and 11-12 in 
ELA and all conceptual 
categories in mathematics at 
the high school level). 
Findings from this study are 
intended to inform 
discussions by various 
consortium groups in terms 
of specifically which content 

X 
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plussed" high school 
standards define the 
minimum college- and 
career-ready finish line for 
all students. However, no 
where in the Common Core 
State Standards document is 
it mandated that these 
learning expectations be 
completed by the 11 th grade. 
If SBAC is planning to 
develop an 11 th grade 
assessment for high school 
mathematics, the SBAC 
states need to come to 
agreement on which of the 
high school standards we 
expect all students to get 
through by at least the end 
of 11 th grade. This is a 
critical issue that the SBAC 
states need to dialogue 
about and come to a 
decision to inform the test 
development. The CCSS high 
school mathematics 
standards specify the 
minimum EXIT expectations 
for all students (i.e., 
expectations for all students 
to achieve by the end of 
their high school careers). 
However, the report implies 
that students will be 
assessed on achieving the 
finish line by the end of the 
11 th grade, whereas the 
standards clearly define a 
minimum finish line for all 
students by the time they 
exit high school. Clarity 
needs to be provided on this 
issue and the SBAC states 
need to define what can 


is tested and how the 
content should be tested 
(e.g., test design, item 
development work groups). 
It is expected that work 
group such as the Test 
Design work group will 
address the issues raised. 
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truly be expected to be 
mastered by the end of 11 th 
grade. 






19. 

We believe there may be 
typos on chart on page 2 
under "Reading Standards: 
Foundational Skills K-5". 
Should 1-4 not be Grades K- 
1, instead of K-2? Should 3-4 
not be 2-5, instead of 3-%? 

Formative Processes and 
Tools/Professional DevWG 

This correction has been 
made in the report. 




20. 

Depth of Knowledge 
definition. Given the type of 
analysis that was done, it 
would be helpful to know 
what verbs were mapped to 
which DOK for this study. 

Performance Tasks WG 

Appendix C provides a 
description of the 
operational definitions of 
the DOK levels used in this 
analysis. Analysts have 
applied this protocol in 
other similar studies, and as 
a result of training for this 
study, analysts were 
calibrated. 




21. 

The eligible standards in 
Mathematics and ELAthat 
are designated to be 100% 
measurable via "technology- 
enhanced" item types. It 
would be helpful to know 
what the working definition 
of "technology-enhanced" 
item types is for this study, 
since they seem to be able 
to accomplish things that 
even performance tasks 
can't do. 

Performance Tasks WG 

Appendix B presents the 
item type descriptions used 
as working definitions for 
this study's analyses. The 
authors of this report 
anticipate that there will be 
further 

refinement/definition of 
each item type (e.g., work of 
various work groups such as 
item development and 
technology approach). 

X 



22. 

Tables 10 through 22: 

These tables use 
abbreviations for item 
types that are not defined 
locally. While one can 
figure out what these 
mean, for clarity it would 
be better to define these 
in the text. 

While 1 understand the 
use of the codes, they 

Performance Tasks WG 

This has been addressed in 
the final report 
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should be identified 
before the table, not after 
each table in which the 
code appears, or else, the 
actual wording of the 
code should appear to 
reduce confusion. 

These tables also present 
data in a way that is not 
parallel to the way the 
data for the Math 
Standards is presented. 
Why isn't the same detail 
available for both Math 
and ELA? (for example, for 
the ELA standards (tables 
10-22) the comments by 
code are given for each 
standard and grade range, 
whereas for Mathematics 
(tables 23-26) do not 
show this level of detail. 
Some of the detail is 
written in a narrative form 
for Mathematics, but not 
at the same level as the 
information that is 
provided for ELA. (for 
example, "Eight of the 50 
standards received 
Comment Codes 301 and 
308, ...")) 






23. 

Comment #1 - Re-evaluate 
two ineligible standards for 
consideration as eligible 
content. Detailed rationale 
provided below for this re- 
evaluation. 

The standards listed below 
were "judged to be more 
appropriate for classroom 
assessment" (SBAC, in draft, 

Performance Tasks WG 

This has been addressed in 
the final report 
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р. 28). 

G.SRT.l. Verify 
experimentally the 
properties of dilations given 
by a center and a scale 
factor: 

a. A dilation takes a line not 
passing through the center 
of the dilation to a parallel 
line, and leaves a line 
passing through the center 
unchanged. 

b. The dilation of a line 
segment is longer or shorter 
in the ratio given by the 
scale factor. 

8.G.I. Verify experimentally 
the properties of rotations, 
reflections, and translations: 

a. Lines are taken to lines, 
and line segments to line 
segments of the same 
length. 

b. Angles are taken to angles 
of the same measure. 

с, Parallel lines are taken to 
parallel lines. 

The two standards above 
highlight the properties of 
geometric transformations 
as applied to one- 
dimensional objects. If we 
deem them ineligible, we 
will limit our coverage of 
geometric transformations 
to two-dimensional figures. 
Looking at standard 
8.G.l.(b), the following 
examples demonstrate that 
it is assessable given our 
item types. While these are 
not polished, final products. 
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they should illustrate the 
point: 

S What happens to the 
measure of a 45 degree 
angle when it is rotated 
120 degrees? Provide a 
series of steps a person 
could follow to show 
what happens, (open- 
ended) 

S What is the measure of 
a 45 degree angle after 
it has undergone a 120 
degree rotation? 
(selected response or 
grid) 

S Using technology, 
provide three different 
examples to show that 
a rotation does not 
affect an angle's 
measure, (technology- 
enhanced) 

■S Create a company logo 
(hand-drawn or using 
technology) where the 
entire drawing is based 
on a single angle and 
the following statement 
"When an angle is 
rotated, reflected, or 
translated the resulting 
angle is the same 
measure." Develop a 
presentation that 
includes a 

demonstration of how 
your drawing was 
developed from a single 
angle of measure 
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and the series 
of transformations used 
to create your final 
product. (Performance 
task) 






24. 

1 do not generally disagree 
with the ratings for DOK 
with respect to the 
mathematics standards. 
Since the purpose of the 
DOK analysis was not clearly 
defined, however, 1 want to 
be certain that these ranges 
of cognitive complexity do 
not prevent the eventual 
test items from extending to 
higher levels of complexity 
then those assigned to the 
individual standards. 

Perhaps an elaboration on 
the purpose and eventual 
use of these particular 
ratings would help alleviate 
any concerns that the 
mathematics standards are 
heavily loaded on the 1 & 2 
levels. 

Performance Tasks WG 

The intention of the DOK 
analysis will be made clearer 
in the report. 

We suggest follow up by 
consortium's work 
groups/committees to 
address the concern 
regarding higher levels of 
DOK being assessed (e.g., 
test design, item 
development, performance 
tasks, content specifications) 

X 



25. 

It would be very helpful to 
clearly define and highlight 
the differences between the 
item types that fall under 
the "Extended constructed- 
response" umbrella - 
"gridded-response, short- 
answer, open-ended, open- 
response, or constructed- 
response" (SBAC, in draft, p. 
11) 

Performance Tasks WG 

We suggest follow up by 
consortium's work groups 
(e.g., test design, item 
development, performance 
tasks, accessibility and 
accommodations; 
technology approach, 
administration) (Related to 
8, 39, 72, 88, 89, 128, and 
134) 

X 



26. 

Correction: Writing Standard 
#9 begins at grade 4, NOT 
grade 3. (p. 2) 

Performance Tasks WG 

This correction has been 
made 




27. 

Question/Clarification: Is 
there an item type labeled 
"constructed response" 

Performance Tasks WG 

This has been addressed in 
the final report 
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apart from extended 
constructed response or 
technology-enhanced 
constructed response? That 
is, are there shorter or less 
complex constructed 
response items planned that 
are not TEs? 

Note commentary under 
Test Format: "The 
selected response and 
CONSTRUCTED RESPONSE 
sections of the summative 
assessment will be 
administered as a 
computer-delivered 
adaptive assessment." 
Does that statement refer 
to extended constructed 
response? Technology- 
enhanced constructed 
response? Both? (p. 6) 






28. 

Are the on-demand writing 
prompts part of the adaptive 
engine? (p. 6) 

Performance Tasks WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 
work groups: test design; 
item development; 
accessibility and 
accommodations; 
administration; technology 
approach. 

X 



29. 

The construction of the ELA 
Conceptual Task (p.35) 
trivializes the writing 
component of the proposed 
reading/writing task. Writing 
about content does not in 
itself guarantee the 
measurement of writing. 
Asking students to 
"present an analysis in 
well-organized 
paragraphs," does not 

Performance Tasks WG 

The examples are intended 
to illustrate how to navigate 
and use the information 
from the analysis in creating 
a task— consideration of the 
content standards, 
comment codes, DOK, etc. 
vis-a-vis a specified 
conceptual task. The 
examples appear to serve 
their purpose, and we 
encourage the work groups 

X 
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mean coverage of 
W.5.2.or W.5.4. The task 
implies writing will be 
measured by "having well- 
organized paragraphs." 
That was not the intent of 
W.5.4. Asking students to 
write an analysis does not 
signal the richness of a-e 
ofW.5.2. 

W.5.2 and W.5.4 
present essential 
elements in a piece of 
writing. If the intent is 
to measure both 
reading and writing 
standards, then the 
entirety ofW.5.2 and 
W.5.2 would need to be 
signaled in the task 
(e.g., "write an essay 
explaining your analysis 
for your teacher" and 
adding W.5.5, e.g.," be 
sure to plan, revise, and 
edit your essay"). 

Rather than looking at 
eligible standards 
individually, might some 
standards be better 
measured if clustered 
together? W.5.2, W.5.4, 
and W.5.5 would make 
an ideal cluster if we are 
interested in measuring 
writing meaningfully. 
The advantage of 
performance tasks for 
writing would be the 
opportunity to provide 
time for quality revision 
and editing— which is 
rarely, if at all possible, 
on short (e.g., 30- 


to use the general process 
illustrated, as appropriate, 
for designing assessment 
tasks (e.g., item 
development, performance 
tasks, formative processes 
and tools/professional 
development). (Related to 
65 and 70) 
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minute) on-demand 
writing prompts. 

Similar comments could be 
made about the reading 
component of the task. 1 
realize this is only a 
conceptual task to illustrate 
how to use eligible content 
data in task development; 
however, if we have one 
opportunity to illustrate a 
performance task in this 
document, wouldn't a 
carefully crafted example 
that illustrates the potential 
of a performance task to 
measure complex thinking 
across standards be a good 
idea? How about a sample 
task that is not the typical 
write-about-what-you-have- 
read task? 






30. 

Consideration: Please 
consider changing the 
wording of comment #309 
to read: Statements in this 
standard inform the 
complexity of text and 
selection of content area 
texts appropriate for this 
assessment. That wording 
would be a better fit for 
ELA Reading standard for 
all grades (p. 48). 

Performance Tasks WG 

This has been addressed in 
the final report 
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31. 

Depth of Knowledge 
classification: While "use" 
is a verb that indicates 
Writing Level 1 in the DOK, 
some use of punctuation 
requires more mental 
processing when sentence 
structures are more 
complex (p. 46). Is the 
Depth of Knowledge 
classification dependent 
only on the verb used? 

Performance Tasks WG 

No, DOK classification is not 
only dependent on the verb 
used. 

This has been addressed in 
the final report 




32. 

Comment #7— 
Considerations: Does 
measuring only a portion of 
a standard sometimes put at 
risk the fidelity of that 
standard? Are some 
standards better assessed in 
clusters? What role does the 
item type play in 
determining whether 
clusters are appropriate? 

Performance Tasks WG 

Response to this question is 
beyond the scope of this 
project/report. Suggest 
follow up by consortium's 
work groups/committees: 
test design; item 
development; content 
specifications. Also suggest 
discussion with consortium's 
TAC. 

X 


X 

33. 

Clarification needed: Does 
"eligible" mean that it will 
be included or considered 
for inclusion? For Grades 3-8 
Mathematics, all but one 
standard is eligible. This is 
an enormous amount of 
content to include on a 
summative test 

Item DevWG/state 1 

This has been addressed in 
the final report 




34. 

p. 5, first sentence in section 
"Purpose of the Summative 
Assessment" 

Can "full range" be clarified. 
Does this mean each and 
every standard will be 
assessed? Does this mean 
that all components of a 
standard need to be 
assessed? For example, 
standards that have a 
measurable code of 302 "At 

Item Dev WG/state 1 

This has been addressed in 
the final report 
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least a portion of this 
standard is best measured 
by in-person observation," 
must those be assessed too? 






35. 

p. 6, Item Types, Extended 
Constructed Response (and 
subsequent references to 
"extended constructed 
response") 

Terminology Clarification: 
Suggestion - "Constructed 
Response" is the more 
general term. In CT 
"extended constructed 
response" is a type of 
constructed response that 
requires the student to write 
a first draft in ELA or solve a 
multi-step math problem 
that has more than one 
possible answer or more 
than one solution method. 
"Constructed response" is 
the general term that 
includes these plus short 
answer, grid-in and short 
answer with an explanation. 
"Constructed response" is 
used in the Test Format 
section below. 

Item Dev WG/state 1 

This has been addressed in 
the final report 




36. 

p. 7, Learnable within a 
school year 

Clarification needed: The 
content may be learnable 
within the school year but is 
it learnable before the 12 
week testing window? For 
mathematics, this means 
that the entire grade-level 
content is eligible to be 
assessed 3 months before 
the end of the year. 

Item Dev WG/state 1 

This has been addressed in 
the final report 
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37. 

p. 8, second sentence under 
#3 

Clarification needed: Does 
this mean that a standard 
that has a measurable code 
of 302 is not measurable via 
on-demand tasks? None of 
these in math were rated as 
not measurable. In these 
cases, only part of the 
standard can be assessed. 
This goes back to my 
question about "measuring 
the full range of the CCSS." 

Item Dev WG/state 1 

This has been addressed in 
the final report 




38. 

p. 11 at top: "analysts 
developed to decision rules" 
Typo? 

Item Dev WG/state 1 

This correction has been 
made 




39. 

p. 11, "Mathematics Only", 
#1 

What is the difference 
between these three types 
of constructed responses? 

Item Dev WG/state 1 

Response to this question is 
beyond the scope of this 
project/report. 

Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). (Related to 
8, 25, 72, 88, 89, 128, and 
134) 

X 



40. 

p. 39, #4, bullet 3 
In appendix E page 6, this 
standard is indicated as 
eligible. 

Item Dev WG/state 1 

This correction has been 
made 




41. 

The purpose of the 
document was clear and 
stated at the beginning as 
expected; however, a few 
formatting changes would 
enhance the readers' 
understanding of the entire 
document: 

A statement identifying 

the intended audience 

State 2 

This has been addressed in 
the final report 
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Inclusion of a table of 
contents 

A clear delineation of 
sections through more 
variation in headings of 
sections and subsections 
(e.g., font size and style, 
indentation/centering. 






42. 

The Table 2 example of the 
Grade 5 learning progression 
in reading establishes the 
relationship between anchor 
standards and K-12 
standards; perhaps a similar 
table could be used for 
mathematics, where the 
relationship of the CCSS and 
the CCR anchor standards is 
not as explicit. 

State 2 

This has been addressed in 
the final report 




43. 

Throughout the document, 
codes were used that were 
either not explained or 
explained after they had 
been used in a table/chart: 

• The protocol for coding 
the ELA standards p. 3 
and pp. 17-24 have no 
explanation. It is a 
complicating step for 
the reader to stop and 
figure this out. 

• The item types listed 
on p. 6 and 8-9 should 
have the abbreviations 
that are used later in 
the tables for ELA and 
mathematics. 

• The item types on p. 6 
should include "oral 
response," which is 
listed as an item type in 
the ELA tables. 

• "Oral response" should 

State 2 

This has been addressed in 
the final report 
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be added to Item Types 
Appendix B. 

• On p. 17, paragraph 3, 
the appendix letter 
should be included 
after item types 
(Appendix B), comment 
codes (Appendix D), 
and DoK range 
(Appendix C). (Note: a 
table of contents might 
render this 
unnecessary.) 






44. 

In Item Types Appendix B, 
two content questions 
arose: 

• ELA- How does the 
ECR differ from the 
WP? ECR appears as an 
item type for reading 
standards only and the 
WP for writing and 
language standards. 
Could a clarification be 
added to the quoted 
descriptions in 
Appendix B? 

• Mathematics - items 
that are labeled TE will 
need to be assessed 
with intuitive 
technology equipped 
with a particular 
interface that includes 
mathematical symbols, 
as well as the capability 
for students to create 
models and graphs. 
That is not clear in the 
quoted description. 

State 2 

We suggest follow up by 
consortium's work groups 
(e.g., test design, item 
development, performance 
tasks, accessibility and 
accommodations; 
technology approach, 
administration) 

X 



45. 

The methodology used was 
clear, and we recognize that 
this portion of the work is 

State 2 

This has been addressed in 
the final report 
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complete. The Coding 
Protocol process prompted 
some concern; i.e. ; that the 
two analysts per content 
area did not code the CCSS 
independently before 
reaching consensus, rather 
than coding by a single 
analyst, with the other 
reviewing the coding and 
then reaching consensus. 

While we question the 
number of raters used, 
we appreciate their 
expertise and wide 
experience which 
includes having been 
classroom teachers. 






46. 

One small suggestion: On 
first reading of the 
"assessable" criteria, the 
description might lead to 
misconception that content 
has been deemed not 
eligible because of 
limitations in our technology 
(remember that old 
assessment saw about 
"Measuring what we value, 
not valuing what we can 
measure). Please add clarity 
to the criteria to make it 
clear that this is not why 
some content has been 
judged as not eligible. 

State 2 

This has been addressed in 
the final report 




47. 

ELA Findings: 

a. Tables 10-22 are 
organized by "strands 
within strands," yet the 
strands do not appear 
prior to these tables. It 
would be helpful to 
have a chart of these 

State 2 

a. This has been addressed 
in the final report 
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strands within strands 
similar to the 
mathematics 
organization chart in 
Table 5. 

b. Perhaps this question 
is premature, but given 
that standards at both 
grades 9-10 and 11-12 
have been identified as 
eligible content for the 
summative 

assessment, how does 
this fit with the idea of 
a single summative 
assessment at the HS 
level in ELA? 

c. Literacy in 
History/Social Studies, 
Science and Other 
Technical Subjects 
standards are not 
mentioned until 
Appendix A, p. 38. 
These standards are 
included as part of the 
ELA standards, but 
there is no explanation 
for the omission. These 
standards have been 
widely deemed a major 
step in increasing 
literacy in all content 
areas, but there will be 
no "teeth" in them if 
they are not included 

in the summative 
assessment. We do not 
even see them in the 
document as eligible 
content to be assessed 
by performance 
assessments. 


b. Response to this question 
is beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work groups: 
test design; item 
development; 
administration 

c. This has been addressed 
in the final report 

X 
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48. 

Mathematics Findings: 

State 2 






a. Fleavy emphasis on 


a-c. Implications for test 

X 




DoK levels 1 + 2 for all 


design and development are 





grades and domains. 


beyond the scope of this 





b. Grades 3-6 are very 


project/report. 





light on DoK 3. 


Suggest follow up by 





c. DoK Levels 2-3, 2-4, 


consortium's work 





and 3-3 appear to be 


groups/committees: test 





heavily weighted 


design; item development; 





toward the domain of 


content specifications 





geometry. 

d. There is no explanation 







of why Mathematical 


d. This has been addressed 





Practices standards 
were omitted from 
eligible content, nor is 
there any mention of 
an Oral Response item 
type that could be used 
to assess a 

Mathematical Practice 
standard such as, 
"Constructing a viable 
argument defending 
your solution to a 
problem." 


in the final report 





e. No mention of the 


e. This has been addressed 





relationship between 
the CCR standards for 
mathematics and the K- 
12 mathematics 
standards, as seen in 
ELA. 


in the final report 





f. Our math specialists 


f. This issue is beyond the 

X 




were particularly 


scope of this project/report. 





concerned about the 


Suggest follow up by 





limited emphasis on 


consortium's work 





mathematics practice 


groups/committees: 





and modeling, two 


formative processes and 





areas that should 


tools/professional 





advance our instruction 


development; content 





in mathematics. Our 
concern is that schools 
may only emphasize in 


specifications 
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instruction what we 
emphasize in 
assessment, which 
would be very 
unfortunate in the case 
of modeling and math 
practices. 






49. 

p. 6, Item Types, 
"Performance Tasks" 

...two at EACH OF grades... 

Test Design WG/state 2 

This correction has been 
made 




50. 

p. 10, Coding Protocol 
Did the second analyst first 
code the standards 
independently? That seems 
a bit unclear. 

Test Design WG/state 2 

This has been addressed in 
the final report 




51. 

p. 10, Coding Protocol 
First mention of a content 
lead. Perhaps a bit more 
explanation of their role? 
Was their training more in- 
depth or just have different 
experiences? What 
differentiates a content lead 
from a rater. Same with 
project leads below. 

Test Design WG/state 2 

This has been addressed in 
the final report 




52. 

p. 14, ELA Findings, Eligibility 
(following table 6) 

Breaking down eligibility in 
this way is a bit confusing. 
Eligibility = learnable + 
expected + measurable. The 
table works well but the 
description in the text in the 
paragraph after the table 
and bulleted list is a bit 
cumbersome. And then the 
next section goes on to only 
discuss eligible standards 
rather than 

measurable/eligible. Seems 
better to just go with 
discussion of eligible. 

Test Design WG/state 2 

This has been addressed in 
the final report 
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53. 

p. 15, in text following Table 
7 

Is this reference to next 
section correct? Seems that 
the discussion of item types 
by CCR anchor standard 
comes a section (or 2?) 
away, after DOK. 

Test Design WG/state 2 

This has been addressed in 
the final report 




54. 

In the Appendix (Guidelines) 
page 38, Question 1, the 
answer regarding 
Mathematics at the High 
School level suggests that all 
six content standards would 
need to be assessed over a 
multiple-year period. There 
is no suggestion as to what 
the assessment design 
would then look like- would 
it be helpful to add some 
comments to distinguish 
between an End-of-Course 
measure and End of 
Mathematics Pathway 
summative? 

Reporting 

The assessment design and 
refined distinctions between 
End-of-Course measures and 
End of Mathematics 
Pathway summative 
assessments are beyond the 
scope of this project/report. 
Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
content specifications 

X 



55. 

page 9, DOK is not specified 
as an eligibility criterion for 
selection of standards for 
inclusion in the assessment. 
My review of the 
accompanying excel 
workbook listing the criteria 
of each CCS suggests there 
are an abundance of DOK 1 
and DOK2 standards, but 
very few DOK3 and DOK4. If 
we truly believe that the 
assessment design must 
provide evidence of student 
learning and mastery in the 
higher DOK levels, should 
we list as a criterion that 
there should be a certain 

Reporting 

An outcome of this study is 
to provide a description of 
the CCSS. Responses to the 
specific issues/questions 
raised are beyond the scope 
of this project/report. 
Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
content specifications, as 
well as with the 
consortium's TAC. 

X 


X 
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percentage of DOK 1-2-3-4- 
rated Standards at each 
level of the Math 
assessment and ELA 
assessment? The discussion 
on page 17 begins to get at 
the distribution of DOK 
items, but needs to be 
translated into a guideline 
for numbers or percentage 
of items at each DOK level 
and at each grade. Both ELA 
and Math have the majority 
of items rated as DOK1, 
DOK2 and DOK3. The 
discussion on page 32 helps 
crystallize both the concern 
and the challenge to select 
appropriate item types to 
address the DOK issue. 

1 would like to make a 
similar comment about a 
proposed percentage of the 
various item types seen as 
desirable for the summative 
test design, especially as the 
assessment design changes 
from elementary grades to 
high school grades 






56. 

More explanation of the 
balanced assessment 
system, including 
interim/benchmark 
assessment, formative 
assessment, and summative 
assessment is required. On 
first reading of the 
document in hand, 1 cannot 
see how the summative 
assessment, which students 
may take twice during the 
testing window, which 
occurs within the last 12 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; reporting; formative 
processes and 
tools/professional 
development; accessibility 
and accommodations; 
content specifications 

X 
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weeks of the instructional 
year (p. 6) can "produce 
instructional^ useful 
information available 
throughout the instructional 
year to help guide and 
support differentiated 
instruction." (p. 5) 






57. 

Why, in Appendix A, are only 
grades 9-10 and 11-12 
considered for eligibility on 
the summative assessment? 
(see p. 38) [Note: 1 moved to 
Appendix A because the 
methodology section of the 
document stated that the 
guiding principles and 
considerations are 
presented in the appendix.] 

Reporting 

Grades 9-10 and 11-12 are 
the high school ELA grades. 
For mathematics, all six 
conceptual categories are 
evaluated at the high school 
level. These were 
determined by (1) the SBAC 
proposal and (2) discussion 
with the consortium's 
Executive Committee. 




58. 

Page 6, Testing Window. 
"The summative assessment 
administered within 12 
weeks of the end of the 
instructional year." Is it 
necessary to clarify the 12 
week time frame? Is the 
window open for 12 weeks? 
Is testing to take place 
sometime within the 12 
weeks? 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work group: 
administration 

X 



59. 

Page 43, Performance Task. 
Please clarify if there are to 
be two performance tasks 
for ELA and two 
performance tasks for math 
per grade for a total of four 
performance tasks per 
grade. Or, will there be as 
stated in the document, 
"...two rigorous 
performance events for ELA 
and mathematics." 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work groups: 
test design; item 
development; performance 
tasks 

X 
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60. 

The phrase "eligible 
content" carries with it a lot 
of misconceptions. Clearly 
defining what is meant by 
"eligible" within the purpose 
section would be helpful. 
The term in the past has 
meant for teachers the 
content that is they will be 
held accountable for. The 
math analysis really shows 
that all the standards are 
assessable, but the 
distinction could be in 
determining what types of 
items could be used, or if 
there would be part of the 
standard that wouldn't be 
assessed (e.g. comment 
308). 

Distancing "eligibility" from 
the idea of "what standards 
are assessed" would be 
helpful (since all standards 
should be assessed 
somehow). Perhaps 
emphasizing the analysis 
determines how the 
standard could be assessed 
and at a specified depth of 
knowledge would be better. 

Reporting 

This has been addressed in 
the final report 




61. 

The use of an "on-demand" 
task (section 3, p.8) is not 
clear in context of the item 
descriptions given earlier on 
page 6. Which of the items 
types listed on page 6 would 
be considered "on- 
demand"? It is confusing 
since the use of "on-demand 
writing prompts" is the only 
reference to the term, and it 
seems like an ELA-specific 

Reporting 

This has been addressed in 
the final report 
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term in this context. Either 
drop the term "on-demand" 
(and then specify the items 
types you are referring to in 
section 3 (p.8)), or clearly 
define which items in the 
"item type" on page 6 would 
be considered "on- 
demand". 






62. 

Better defining the depth of 
knowledge descriptors 
would be helpful (p.9). 1 
know it is referenced, but it 
seems like 1 shouldn't need 
to look up the article to 
figure out what you are 
referring to in this context. 
Also, nearly all the math 
standards were not given a 
level 4 DOK rating. What 
would qualify an item as 
extended thinking then? 
Perhaps an example, or 
exemplar of an item would 
help illustrate these 
differences. 

Reporting 

Appendix C provides a 
description of the 
operational definitions of 
the DOK levels used in this 
analysis. The citation, Webb 
(2005), is provided to 
reference the source 
document from which the 
description of the DOK levels 
was excerpted. 

Examples of items 
measuring content at DOK 4 
is beyond the scope of this 
project/report. Work by 
consortium work 
groups/committees (e.g., 
item development, 
performance tasks, content 
specifications) should yield 
examples of items that 
measure content at DOK 4. 

X 



63. 

Eligibility is better defined 
on page 14, and would be 
helpful to include earlier. It 
seems that the term is 
referring to the eligibility to 
the online summative test 
only. It would be good to 
clarify this early in the 
document. 

Reporting 

This has been addressed in 
the final report 




64. 

It is not clear how DOK 
levels (table 25, p.30) were 
determined for standards 
that demonstrated multiple 
components (table 26, p.30). 

Reporting 

This has been addressed in 
the final report 
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Items that are 1-1, 2-2, and 
3-3 are clear, but how would 
a 1-2 get coded as a single 
number? Or a 1-3? Table 26 
makes sense to me, but how 
you get table 25 from the 
data in table 26 is not clear 
and seems subjective. How 
was this process 
operationalized by the 
raters? 






65. 

Not sure what the purpose 
of the Examples of use of 
Data section would be (p.35) 
for the reader. Is this an 
appendix type of item? It 
seems like it is future 
oriented, which makes it not 
clear how it relates to what 
has been done. Why is the 
table in step 3 blank? Could 
there be an example of what 
this may look like? 

Reporting 

The examples are intended 
to illustrate howto navigate 
and use the information 
from the analysis in creating 
a task— consideration of the 
content standards, 
comment codes, DOK, etc. 
vis-a-vis a specified 
conceptual task. The 
examples appear to serve 
their purpose, and we 
encourage the work groups 
to use the general process 
illustrated, as appropriate, 
for designing assessment 
tasks (e.g., item 
development, performance 
tasks, formative processes 
and tools/professional 
development). (Related to 
29 and 70) 

X 



66. 

The term "eligibility" needs 
to be operationalized better 
in the introduction. 
Anticipate possible mis-use 
of such a document to be 
used by people to figure out 
what is the minimum they 
need to teach to get by. 
Emphasizing that all 
standards should be 
assessed in the classroom 
could help as well as 

Reporting 

This has been addressed in 
the final report 
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emphasizing that process of 
determining item types and 
depth of knowledge are also 
part of the eligibly process. 






67. 

Clarify what is meant by 
"on-demand" task on page 
8, and be sure it is 
consistent with the item 
type table on page 6. 

Reporting 

This has been addressed in 
the final report 




68. 

Definitions and examples of 
DOK descriptors on page 9 
would be helpful. Especially 
what would it take to be a 4 
since only one made this 
rating. 

Reporting 

Appendix C provides a 
description of the 
operational definitions of 
the DOK levels used in this 
analysis. 




69. 

Clarify how a standard is 
given a single DOK rating for 
table 25 (p.30) is 
determined when they have 
multiple DOK rating (e.g 1-2, 
1-3) found in table 26. That 
is, it would help to 
understand how an item 
would go from table 26 to 
25. Without this explanation 
this process seems very 
subjective. 

Reporting 

This has been addressed in 
the final report 




70. 

Include more explanations 
to the "use of Data section" 
(p.35). Examples for the 
table in step 3 may help. 

Reporting 

The examples are intended 
to illustrate howto navigate 
and use the information 
from the analysis in creating 
a task— consideration of the 
content standards, 
comment codes, DOK, etc. 
vis-a-vis a specified 
conceptual task. The 
examples appear to serve 
their purpose, and we 
encourage the work groups 
to use the general process 
illustrated, as appropriate, 
for designing assessment 
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tasks (e.g., item 
development, performance 
tasks, formative processes 
and tools/professional 
development). 

(Related to 29 and 65) 




71. 

a. If the goal for interim 

Reporting 

a. Response to this question 

X 

X 

X 


assessments is to 


is beyond the scope of this 





identify the level at 


project/report. 





which a student is 


Suggest follow up by 





performing successfully. 


consortium's work 





even if that is above or 


groups/committees: test 





below grade level, then 


design; item development; 





shouldn't we be 


formative processes and 





developing items to 


tools/professional 





reflect standards in K-2 


development; accessibility 





and 11-12? That is not in 


and accommodations; 





the plans, mainly 


content specifications. 





because there is not 


Discussion with the 





enough funding in the 


consortium's Executive 





grant. It seems we might 


Committee and TAC also is 





want to recommend this 


recommended. (Related to 





development occur, and 


74 





seek funds elsewhere 
for that work. (Linda 
Darling Hammond says 
there are foundations 
waiting to help fund 
various things). 


and 82) 





b. Related to that, the 


b. Response to this issue is 

X 

X 

X 


intent is to only include 


beyond the scope of this 





items that are "available 


project/report. The project's 





to all students." That is 


analysts were tasked with 





why they are not 


analyzing the grades 





included STEM items in 


specified in the report for 





11-12. 1 think it should 


ELA and mathematics. 





still be developed at that 


Additionally, the scope of 





level if we say this is 


this project/report was 





supposed to give 


limited to the summative 





information about 


assessment— therefore, 





college and career 


consideration of content 





readiness. Otherwise 


eligibility was consistent 





the ceiling will be too 


with the intent of the 





low. 


summative assessment 
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which is intended for all 
students, since the 
summative assessment is for 
accountability. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
content specifications. 
Discussion with the 
consortium's Executive 
Committee and TAC also is 
recommended. (Related to 
74 and 82) 




72. 

If this work is meant to 
identify the types of items 
that would best 
demonstrate attainment of 
each standards, 1 want to be 
sure that there has been 
enough thought given to the 
"best item types." It could 
be that we default to 
multiple choice too 
frequently and that could 
limit how we think of 
development of assessment 
types later. 

Reporting 

Determination of the "best 
item type" for each standard 
is beyond the scope of this 
study. More/refined 
definition of the item types 
is needed. 

Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). 

(Related to 8, 25, 39, 88, 89, 
128, and 134) 

X 



73. 

1 think that "speaking and 
listening" is going to lend 
itself best to the classroom 
performance tasks that are 
to be part of the 
assessment, although 
listening could also be done 
via computer with 
headphones. The PARCC 
consortium is 

Reporting 

This issue requires follow up 
by the consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
technology approach; 
accessibility and 
accommodations; formative 
processes and 
tools/professional 

X 

X 

X 
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recommending that 
speaking and listening only 
be assessed by interim 
assessments and not 
included in summative 
testing, but 1 can't imagine 
that USED will allow a major 
section to be excluded. We 
have to figure out how to do 
something for that area in 
the summative assessment. 


development; content 
specifications. Discussion 
with the consortium's 
Executive Committee and 
TAC also is recommended. 




74. 

if there is the intention to go 
off grade level with the 
adaptive test, then shouldn't 
standards that are NOT for 
all students still be included? 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
content specifications. 
Discussion with the 
consortium's Executive 
Committee and TAC also is 
recommended. (Related to 
71 a and b and 82) 

X 

X 

X 

75. 

Page 5 mentions 
differentiated instruction; 
when 1 look at the SBAC 
glossary, this is not defined. 
It might not be a big issue, 
but recently 1 have run into 
challenges with how this 
term is used in the field and 
am wondering if SBAC has 
come to consensus on a 
definition for this term. 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work 
groups/committees: 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
content specifications. 
Discussion with the 
consortium's Executive 
Committee and TAC also is 
recommended. 

X 

X 

X 
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76. 

Page 6 discusses that 
portions of the summative 
assessment will be 
administered as a computer- 
delivered adaptive 
assessment; how will this 
work for kids that can't 
access computer-delivered 
materials? 1 am sure 
accommodations have been 
considered. ...should this be 
addressed here? 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work groups: 
test design; item 
development; technology 
approach; accessibility and 
accommodations. Discussion 
with the consortium's 
Executive Committee and 
TAC also is recommended. 

X 

X 

X 

77. 

Beginning on page 17 there 
are comments codes used in 
the provided tables (301, 
308, 305 and so forth) 
however the specific 
meaning of these codes are 
not provided until after the 
table has shared the 
information. It might be 
more helpful to provide a 
key or identify the meaning 
of these codes prior to 
presenting them in the 
tables. 

Reporting 

This has been addressed in 
the final report 




78. 

Page 15 typo, should read- 
245 via performance tasks 

Reporting 

This correction has been 
made 




79. 

Mathematics Findings: not 
broken down by domain like 
the ELA findings- should be 
consistent 

Reporting 

This has been addressed in 
the final report 




80. 

The Standards for 
Mathematical Practice are 
not addressed in any of the 
materials. 1 know that they 
are very difficult to assess, 
but they are also central to 
what is considered to be 
mathematical proficiency. 1 
think this is a serious 
omission. 

Reporting 

This will be addressed as 
appropriate in the final 
report and/or in the final 
version of this response 
document. 
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81. 

The memo has a flaw when 
describing project 
background (red in quote 
below). The eligible content 
is not wholly defined in 
Algebra and Geometry. This 
statement limits the intent 
of the standards even 
though throughout the 
document there is 
consistent reference to the 
six conceptual categories 
which includes topics 
beyond algebra and 
geometry. Referring only to 
the traditional algebra- 
geometry sequence stifles 
efforts in moving toward 
integrating standards. 

The eligible content will be 
defined for ELA and 
mathematics in grades 3-8 
and high school (HS). 
Regarding HS: for ELA this 
means grades 9-10 and 11- 
12; for mathematics, this 
means Algebra and 
Geometry — all six 
conceptual categories (Note: 
Modeling is integrated into 
five of the conceptual 
categories." 

Reporting 

The reference to Algebra 
and Geometry in the memo 
that provides context and 
instructions for the review 
of the draft report is an 
artifact of a previous, more 
limited, reference to 
mathematics content. The 
references in the report to 
the six conceptual 
categories is correct. 




82. 

there needs to be some kind 
of a plan for K-2 and STEM 
item development. 

Reporting 

Response to this question is 
beyond the scope of this 
project/report. The project's 
analysts were tasked with 
analyzing the grades 
specified in the report for 
ELA and mathematics. 
Suggest follow up by 
consortium's work 
groups/committees: test 

X 

X 

X 
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design; item development; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
content specifications. 
Discussion with the 
consortium's Executive 
Committee and TAC also is 
recommended. (Related to 
71 a and b and 74) 




83. 

For the ELA and Math 
Findings sections, even if 
there were NO eligible 
content for these sections, 
tables should be included 
with "N/A" present. It would 
make it more inclusive. 

Item Dev/State 2 

This request has been 
noted; however, the 
decision was made to 
streamline the report as 
much as possible and to not 
add this table to the report. 




84. 

For the Summary of Coding 
tables (e.g.. Table 10), 1 
would like to see one single 
overall table (in addition to 
the tables within the 
discussion). It would make it 
easier to get a global 
picture. 

Item Dev/State 2 

This request has been 
noted; however, the 
decision was made to not 
add this table— there are 
content and strand 
differences that would make 
such a table difficult to 
interpret and possibly 
misrepresent information. 




85. 

Re: What additional 
guidance for organizing and 
using the data and findings 
should be included in this 
report's discussion of 
considerations and 
implications? 

1 would appreciate seeing an 
outline of a global 
spreadsheet which will allow 
us to start making decisions 
about which items and in 
what quantities will be 
needed. 

Item Dev/State 2 

Response to this question is 
beyond the scope of this 
project/report. 

Follow up is needed by 
consortium's work groups: 
test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach 

X 



86. 

Re: What questions do 1 
have or decisions do 1 need 

Item Dev/State 2 
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Work Group(s) 
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u 

< 

H 


to make that could be 
informed by the outcomes 
of this study? 







a. Proportions of item 


a. Response to this question 

X 




types supported by 


is beyond the scope of this 





this analysis 


project/report. 

Follow up is needed by 
consortium's work groups: 
test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach 





b. Are certain item 


b. Response to this question 

X 




types better ranged 


is beyond the scope of this 





for specific DoKs 


project/report. 





within standards. 


Follow up is needed by 
consortium's work groups: 
test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach 




87. 

1 know the application only 

Item Dev/State 2 

Response to this question is 

X 




defined extended 


beyond the scope of this 





constructed response. We 


project/report. 





have discussed recently 


Follow up is needed by 





about also having a short 


consortium's work groups: 





constructed response 


test design; item 





category. How would that fit 


development; performance 





into this document? - 


tasks; accessibility and 





related to pg. 6, Item Types 


accommodations; 
technology approach 




88. 

Related to pg. 8, Item Type - 

Item Dev/State 2 

More/refined definition of 

X 




knowing that the analysis 


the item types is needed. 





was looking at DoK and Item 


Follow up is needed by 





Types across the range of 


consortium's work groups 





standards, can any insight 


(e.g., test design; item 





be provided into what items 


development; performance 





could tap which levels of 


tasks; accessibility and 





DoK for standards? 


accommodations; 
technology approach; 
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administration). 

(Related to 8, 25, 39, 72, 89, 
128, and 134) 




89. 

Related to pg. 11, 
Mathematics only point 1 - 
this interpretation of 
extended constructed 
response items is pretty 
broad. This may answer the 
question about short 
constructed response items 

Item Dev/State 2 

More/refined definition of 
the item types is needed. 
Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). 

(Related to 8, 25, 39, 72, 88, 
128, and 134) 

X 



90. 

Pg. 14, bottom of page - 
was there any pattern 
(across grade bands) to the 
standards that were not 
measurable? 1 assume yes. If 
yes, it would be good to 
note what the pattern was. 

Item Dev/State 2 

This has been addressed in 
the final report 




91. 

Pg. 19, top paragraph, "since 
assessing these standards 
would involve knowing the 
student's point of view." 
Technology enhanced items 
could possibly address this. 
E.g., items of equal difficulty 
that address selected points 
of view. In other words, the 
student could select a point 
of view from some options, 
and then a question tailored 
to that point of view could 
be presented. 

Item Dev/State 2 

This has been addressed in 
the final report 
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92. 

Pg. 20, Reading Standards: 
Foundational Skills, grades 
3-5 - could you add a table 
even if N/A fills the grid. 
Otherwise, this part is hard 
to register as significant, 
when it may be more 
significant than the others. 

Item Dev/State 2 

This request has been 
noted; however, the 
decision was made to 
streamline the report as 
much as possible and to not 
add this table to the report. 
Additionally, very few 
standards are not eligible. 




93. 

Pg. 22, Range of Writing, 
could you add a table even if 
N/A fills the grid. Otherwise, 
this part is hard to register 
as significant, when it may 
be more significant than the 
others. 

Item Dev/State 2 

This request has been 
noted; however, the 
decision was made to 
streamline the report as 
much as possible and to not 
add this table to the report. 
Additionally, very few 
standards are not eligible. 




94. 

Pg. 22, Range of Writing - 
couldn't this be addressed 
via Performance Task items? 

Item Dev/State 2 

Discussion of this report's 
findings is encouraged by 
work groups (e.g., item 
development; performance 
tasks, test design; 
technology approach; 
accessibility and 
accommodations; 
administration) 

X 



95. 

Pg. 23, Summary of Speaking 
and Listening - why can this 
not be addressed with PT 
items? 

Item Dev/State 2 

Discussion of this report's 
findings is encouraged by 
work groups (e.g., item 
development; performance 
tasks, test design; 
technology approach; 
accessibility and 
accommodations; 
administration) 

X 



96. 

Pg. 27, last two paragraphs - 
why could this not be 
addressed via Technology 
Enhanced items? 

Item Dev/State 2 

Discussion of this report's 
findings is encouraged by 
work groups (e.g., item 
development; performance 
tasks, test design; 
technology approach; 
accessibility and 
accommodations; 
administration) 

X 
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97. 

Pg. 29, middle two 
paragraphs - what about 
Technology Enhanced Items 

Item Dev/State 2 

Discussion of this report's 
findings is encouraged by 
work groups (e.g., item 
development; performance 
tasks, test design; 
technology approach; 
accessibility and 
accommodations; 
administration) 

X 



98. 

Pg. 30-Table 25 -the lack 
of DoK 4s is very concerning. 
Because it is so concerning, 
should we verify this finding 
with the writers of CCSS. In 
other words, was this 
intended, or did someone 
(i.e., WestEd analysts or the 
CCSS writers) miss the 
mark? 

Item Dev/State 2 

Analysts were experienced 
with the DOK protocol, and 
they were trained and 
calibrated. 

DOK ratings should be used 
to inform discussion by work 
groups/committees as they 
determine howto best 
measure the standards with 
assessment items (e.g., test 
design, item development, 
performance tasks, 
accessibility and 
accommodations, content 
specifications) 

X 



99. 

Pg. 38 - item 1 - the 
phrasing of this is unclear 

Item Dev/State 2 

This has been addressed in 
the final report 




100. 

Pg. 38 - note to SBAC EC, 
need to address how 
Modeling will be integrated 
through assessment 

Item Dev/State 2 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
technology approach; 
content specifications. 
Discussion with the 
Executive Committee also is 
recommended. 

X 

X 


101. 

Pg. 39 - item 4, first bullet, 
should we address this at 
the high school level as an 
adaptive test? 

Item Dev/State 2 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 

X 

X 

X 
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performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
technology approach; 
content specifications. 
Discussion with the 
Executive Committee and 
TAC also is recommended. 




102. 

Pg. 39 - item 4, 2nd bullet, 
add "beyond the length of 
time allotted for 
performance tasks" 

Item Dev/State 2 

This has been addressed in 
the final report 




103. 

Pg. 40 - item Cl 2 nd bullet - 
should we specify who the 
"other experts" were 

Item Dev/State 2 

This has been addressed in 
the final report 




104. 

Pg. 41 - DoK, this report 
should justify somewhere 
why Webb's DoK was used 
instead of others 

Item Dev/State 2 

This has been addressed in 
the final report 




105. 

Pg. 44 - put Math first in this 
section to follow pattern of 
the rest of the document 

Item Dev/State 2 

This has been addressed in 
the final report 




106. 

Pg. 49 - was there an ELA 
spreadsheet for standards? 

Item Dev/State 2 

Yes 




107. 

Terminology is clear. It 
might be helpful, however, 
to list Item Type acronyms 
(e.g., extended constructed 
response (ECR) in the 
original list of items (pp. 8-9) 
- and in all table headings). 
The acronyms are used to 
indicate eligible item types 
within the tables, but they 
are never connected to the 
item type names. 

Item Dev/State 3 

This has been addressed in 
the final report 




108. 

It might help to have the 
subdivisions of each section 
in CAPS (keep them on the 
left margin). Occasionally 
things seem to run together 

Item Dev/State 3 

This has been addressed in 
the final report 
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(e.g., p. 9, with an 
underline, a numbered 
subdivision, and then a 
main heading). 






109. 

Specific examples are 
helpful, for example on pp. 
3-4 for Language Arts. Could 
something similar be done 
for Math? Later, could that 
same example (s) be used to 
show listed eligible item 
types, comments, and DOK 
range to tie things together 
- and then show how those 
findings could be used (see 
#5 below)? 

Item Dev/State 3 

This has been addressed in 
the final report. Additionally, 
follow up discussions are 
expected to occur among 
members of the 
consortium's work group, 
such as those involved in 
content specifications and 
item development. 




110. 

Mathematics: There is a 
significant number (43) of 
advanced math standards — 
denoted with + — that are 
not expected of all students. 
These standards are not 
included in the summative 
assessment. Why are these 
standards included? Is there 
another advanced 
assessment being planned. 

Item Dev/State 3 

Exclusion of these standards 
for this analysis for the 
summative assessment were 
determined upfront (see 
Appendix A, Guiding 
Principles and 
Considerations). Exclusion 
from this analysis does not 
preclude these standards 
from being addressed by the 
consortium's other 
assessments (e.g., 
formative) 




111. 

Table 27 on page 33 of the 
document lacks clarity. The 
table shows several columns 
where notations are made, 
and the headings of those 
columns are not explicitly 
defined. We can infer that 
"Content Clustering" might 
mean items would be used 
to measure more than one 
standard, but we are not 
sure if that is the accurate 
interpretation. A short 
explanation of that table 
would help add some clarity. 

Item Dev/State 3 

This has been addressed in 
the final report 
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112. 

We did a spot check of a few 
of Appendix E mathematics 
entries and agree with the 
comments. WestEd's 
classification tends to place 
more standards as Recall 
than [our State] Department 
Education would. For 
example, in a number of 
situation [the Department] 
might call a standard 
Skill/Concept whereas 
WestEd denoted it as Recall. 
This is a matter of 
consensus, and 1 don't 
fundamentally disagree with 
them. [The Department] 
would find this classification 
useful if there was SBAC 
agreement regarding usage 
of the Depth of Knowledge 
terms. 

Item Dev/State 3 

Analysts were experienced 
with the DOK protocol, and 
they were trained and 
calibrated. 

DOK ratings should be used 
to inform discussion by work 
groups/committees as they 
determine howto best 
measure the standards with 
assessment items (e.g., test 
design, item development, 
performance tasks, 
accessibility and 
accommodations; content 
specifications) 

X 



113. 

We would like to have 
Appendix E - type document 
with the standards and 
eligible content in column 1 
(in addition to the codes 
currently listed). It is 
awkward to do the cross 
checking. 

Item Dev/State 3 

The data work book can be 
merged with a source table 
containing the text of each 
standard. The data 
workbook also can be sorted 
to organize the information 
according to eligibility. 




114. 

p. 15 - In the first line under 
the chart, change the second 
285 to 245 

Item Dev/State 3 

This correction has been 
made 




115. 

p. 24 - The chart at the 
bottom does not list all 
eligible item types for the 3 
standards, but the comment 
below the chart says it does. 

Item Dev/State 3 

This correction has been 
made 




116. 

p. 29 - The last 2 sentences 
in the last paragraph under 
Comments don't relate to 
comments. 

Item Dev/State 3 

This has been addressed in 
the final report 
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117. 

The volume of eligible 
content for each grade level 
is too much for any one test. 
The weighting of 
standards/eligible content in 
the assessment will be 
important work. With the 
amount of eligible content, a 
system of rotation may be 
necessary to carefully assess 
all standards over a period 
of a few years. 

Item Dev/State 3 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
technology approach; 
content specifications. 
Discussion with the 
Executive Committee and 
TAC also is recommended. 

X 

X 

X 

118. 

From standards and 
curriculum staff: We were 
hoping that with this new 
assessment could get away 
from the term "eligible 
content." We understood all 
of the standards were to be 
thought of as "fair game for 
the common assessment." 
And while we certainly hope 
the assessment will NOT 
attempt to measure them 
all, is there another phrase 
that can be used? We 
always felt denoting "eligible 
content" screamed "teach 
only these standards." 

Item Dev/State 3 

For the purpose of this 
study/report, the term 
"eligible content" will be 
used. Its definition is 
explained within the context 
of this project/report, and 
the intended use of the 
study's findings also is 
explained. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; formative processes 
and tools/professional 
development; accessibility 
and accommodations; 
administration; content 
specifications. Discussion 
with the Executive 
Committee and TAC also is 
recommended. 

X 

X 

X 

119. 

With the features 
mentioned, this assessment 
will require considerable 
investment in hardware for 
schools. 

Item Dev/State 3 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; administration; 
technology approach. 
Discussion with the 
Executive Committee and 
TAC also is recommended. 

X 

X 

X 
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120. 

Some questions arise in 
relation to [State] policies 
(these may not be 
appropriate for the 
requested review) but are 
represented of local 
concerns: 

• For the purposes of 
Essential Skills and 
graduation 
requirements, will 
students be held 
accountable for grade 9- 
10 standards, or 11-12? 

• There is a statement 
that the testing window 
would be within 12 
weeks of the end of the 
instructional year. What 
are the implications 
there for the testing 
window as we currently 
have it in Oregon 
(October - May)? The 
high school assessment 
in ELA specifically refers 
to the fact that the 
learning is based on a 
cumulative process 
across grades. Does that 
imply that a different 
window is provided for 
high school as opposed 
to the other grades, 
which is measuring end 
of year learning? 

• There is a reference to 
the potential 
requirement for six 
performance tasks at 
the high school level, 
each taking up to one or 
two periods. Assuming 
these would be in 

Item Dev/State 3 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
technology approach; 
content specifications. 
Discussion with the 
Executive Committee and 
TAC also is recommended. 

X 

X 

X 
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addition to other 
elements of testing 
(selected response, 
extended constructed 
response, writing 
prompt, technology 
enhanced), exactly how 
much testing would be 
required? How will 
stakeholders respond to 
what appears to be 
more time spent on 
testing? 

• There is an emphasis in 
ELA on technology 
enhanced items; in fact, 
some of the standards 
require technology 
enhanced items in order 
to measure the 
standard. What are the 
implications for 
requiring schools to 
have available and 
adequate technology to 
complete this form of 
testing? 






121. 

The table on p. 16 is a 
concern in that a large 
percentage of the 
assessment will need to be 
measured via extended 
response, performance task, 
writing and oral response. 
How will these be scored? 
What are the ramifications 
for testing; i.e. how long per 
student? 

Test Design/State 1 
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122. 

Concerns about p. 25 about 
portions and parameters for 
assessment - "difficult to 
define for a component" - 
will the assessment only be 
assessing portions of a 
standard? 

Related to the above, the 
comments/coding in 
Appendix D, p. 48 are 
disconcerting as to the 
consistency and strength of 
the standards; i.e. 
progression of learning. 

Test Design/State 1 





123. 

The question that keeps 
coming from teachers in the 
state is; "Who is responsible 
for teaching the 
history/science literacy 
standards? Will they be 
assessed on the reading test 
or the other content areas?" 

Test Design/State 1 

Suggest follow up by 
consortium's work 
groups/committees: 
formative processes and 
tools/professional 
development; content 
specifications. Discussion 
with the Executive 
Committee and TAC also is 
recommended. 

X 

X 

X 

124. 

Page 15 - typo - under chart 
it should be "245 via 
performance tasks" not 285. 

Test Design/State 1 

This correction has been 
made 




125. 

Page 1, Key Question 1 and 
last paragraph: Shouldn't 
interim assessments be 
included here? My 
understanding is that all 
interim and summative 
items will be field tested 
form one pool. Hence, the 
assessable content for the 
summative should be the 
same as the assessable 
content for the interim 
unless there is a plan to 
include additional item 
types for the interim 
assessments. 

Test Design/State 3 

The scope of this 
project/report is limited to 
the summative assessment 
only. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
technology approach; 
content specifications. 

X 
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126. 

Page 6, paragraph: It might 
be helpful to explicitly state 
here that the assessment 
will not be accessible for 
students with severe 
cognitive disabilities. AAs 
that are aligned to CCSS will 
need to be developed. 

Test Design/State 3 

This has been addressed in 
the final report 




127. 

Page 6, Testing Window: 

How long will it take the 
average student to complete 
an assessment for one 
content area considering 
that some items are "to be 
completed over a number of 
days" (p. 8)? How many 
computers will be needed by 
schools (e.g., 1 computer for 
every 5 students)? 

Test Design/State 3 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
administration; technology 
approach; content 
specifications. Discussion 
with the consortium's 
Executive Committee also is 
recommended. 

X 

X 


128. 

Page 6, Item Types: It would 
be helpful to include a 
definition for each of the 
listed item types. Appendix 
B appears to justify the 
inclusion of the various 
types of items, but does not 
include a description of each 
item type. 

Test Design/State 3 

The descriptions provided in 
Appendix B are the item 
type definitions available by 
the analysts for this 
study/report. 

More/refined definition of 
the item types is needed. 
Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). 

(Related to 8, 25, 39, 88, 89, 
and 134) 

X 



129. 

Page 6, Test Format: 1 do not 
understand how it is 
possible to include all the 
item types listed, if the 

Test Design/State 3 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 

X 

X 

X 
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assessment is to be 
adaptive. Will performance 
tasks, extended CR items, 
and writing be scored by the 
computer? Are only the MC 
and technology enhanced 
items administered 
adaptively? (Also see p. 8 
"to be completed over a 
number of days") 


performance tasks; 
formative processes and 
tools/professional 
development; accessibility 
and accommodations; 
administration; technology 
approach; content 
specifications. Discussion 
with the consortium's 
Executive Committee and 
TAC also is recommended. 




130. 

Page 7, Learnable within the 
School Year, last bullet: If N, 
then it should be indicated 
by what grade level the 
content should be learned 
and assessed. 

Test Design/State 3 

Response to this question is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work 
groups/committees: 
formative processes and 
tools/professional 
development; content 
specifications 

X 



131. 

Page 8, number 3: "On- 
Demand-Task" should be 
defined. 

Test Design/State 3 

This has been addressed in 
the final report 




132. 

Page 9, Speaking and 
Listening: Please provide a 
rationale for excluding MC 
items and performance 
tasks. 

Test Design/State 3 

This has been addressed in 
the final report 




133. 

Page 9, DOK: "...analysts 
indicated all applicable DOK 
levels to indicate the full 
range of DOK." — Isn't it 
customary to indicate the 
highest level only? It should 
be understood that 
standards can be assessed at 
levels at and below the 
indicated level (e.g., if DOK = 
2, one could assess at DOK 1 
and DOK 2; if DOK = 3, one 
could assess at DOK 1, 

DOK2, and DOK 3). 1 think it 
would be beneficial to 

Test Design/State 3 

It is not always the case that 
a higher level DOK rating 
(e.g., 3) automatically 
subsumes all lower DOK 
levels (both 1 and 2). 
Therefore, each applicable 
DOK level was noted for a 
given standard. 
Determination of the "target 
DOK" and the DOK level at 
which the standard is 
expected to be generally 
taught is beyond the scope 
of this project/report. 
Suggest follow up by 

X 
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include the target DOK level 
for the standard, meaning 
the DOK level at which the 
standard is expected to be 
generally taught. 


consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
accessibility and 
accommodations; formative 
processes and 
tools/professional 
development; content 
specifications. 




134. 

Page 11, Mathematics Only, 
1: It is necessary to have 
operational definitions for 
all the item types mentioned 
in this document. When one 
reviews the literature, it 
becomes immediately 
evident that there isn't a 
common definition of open- 
response, extended 
constructed response, short 
answer or constructed 
response, etc. How does one 
type differ from the other? 

Test Design/State 3 

More/refined definition of 
the item types is needed. 
Follow up is needed by 
consortium's work groups 
(e.g., test design; item 
development; performance 
tasks; accessibility and 
accommodations; 
technology approach; 
administration). 

(Related to 8, 25, 39, 88, 89, 
and 128) 

X 



135. 

Page 14 and Page 26: It 
would be useful to include 
rationales/justifications/expl 
anations for those standards 
that were judged to be non- 
measureable. 

Test Design/State 3 

This has been addressed in 
the final report 




136. 

Page 15 and 28: Please 
explain what "efficiently" 
means in the sentence: "... 
which could be most 
efficiently used to assess ..." 

Test Design/State 3 

This has been addressed in 
the final report 




137. 

Page 15, Table 7: Include 
ELA in the title -- "....Eligible 
ELA Standards" 

Test Design/State 3 

This has been addressed in 
the final report 




138. 

Page 15, Response Type: It 
would be useful to provide 
the rationales for the 
judgments. 

Test Design/State 3 

This has been addressed in 
the final report 
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139. 

Page 16, Table 8: It would be 
more useful to construct a 
table listing the "expected to 
be taught" DOK level for the 
standards with the 
corresponding percentages. 

Test Design/State 3 

Determination of "expected 
to be taught" DOK is beyond 
the scope of this 
project/report. 

Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
accessibility and 
accommodations; formative 
processes and 
tools/professional 
development; content 
specifications. 

X 



140. 

a. Page 17, Paragraphs 2: 
The description here 
about how DOK levels 
were assigned is different 
from the process 
described on page 9 
("Because some 
standards describe skills 
at multiple levels of 
complexity (e.g., when 
there are multiple skills in 
a standard that could be 
applied at different levels 
of complexity), analysts 
indicated all applicable 
DOK levels to indicate 
the full range of DOK"). 

b. General comment: It 
does not appear to be 
realistic to include a large 
number of DOK 4 items 
on a summative 
assessment. Actually, we 
should probably discuss 
whether DOK 4 items are 
eligible to be assessed on 
the on-demand 
summative assessment. 

Test Design/State 3 

a. This has been addressed 
in the final report 

b. Suggest follow up by 
consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
accessibility and 
accommodations; 
formative processes and 
tools/professional 
development; content 
specifications. 

X 
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141. 

Page 17, Table 10: Codes 
used for eligible items have 
not been introduced (SR = 
Selected Response, etc.). 

Test Design/State 3 

This has been addressed in 
the final report 




142. 

Page 31, paragraph 3: The 
concept of "...not 
measureable..." versus "... 
difficult to test..." should not 
be treated as synonymous. If 
something is "difficult," it is 
still doable. 

Test Design/State 3 

This has been addressed in 
the final report 




143. 

1 don't find the tables 
describing whether each 
standard is measureable by 
various item types useful. It 
would be more useful to 
demonstrate that using the 
various item types for each 
standard would provide 
valuable information over 
and above the information 
that can be provided by 
multiple choice items. For 
example, the question 
shouldn't be whether a 
standard is measureable by 
constructed response items. 
The question should be 
whether the inclusion of 
constructed response items 
for the standard would 
result in obtaining 
information over and above 
the information that is 
provided by the multiple 
choice items. One must keep 
in mind that multiple-choice 
items can be designed to 
measure understanding and 
reasoning skills. 

Test Design/State 3 

The information about 
various item types is 
intended to inform 
discussion by various 
consortium work 
groups/committees. 
Determination of which 
item(s) would yield the most 
meaningful information vis- 
a-vis the purpose of the 
assessment is beyond the 
scope of this project/report. 
We encourage the work 
groups; committees (e.g., 
test design; item 
development, performance 
tasks, formative processes 
and tools/professional 
development; 
administration; reporting; 
content specifications) to 
use the data (examples of 
how to navigate the data 
tables are provided in final 
section of the report) to 
address issues such as the 
one raised. 

X 



144. 

Page 34, last paragraph: 
Committees of state 
curriculum and assessment 
specialists should be 

Test Design/State 3 

Further clarification of the 
intention of this subsection 
has been included in the 
final report. 

X 
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convened to confirm the 
conclusions drawn by 
WestEd. 


This report provides a 
description of the 
standards— the intention of 
the information provided in 
the report is to inform 
discussion; it is not to set 
firm parameters on how the 
standards are to be 
measured. Therefore, it is 
not recommended that the 
conclusions presented in the 
report be 

confirmed/refuted; rather, 
the recommendation is to 
use the descriptions in the 
report (e.g., DOK levels, item 
types, measurability) to 
systematically think through 
what and how content 
should be assessed within 
the specific context/purpose 
of the summative 
assessment. 




145. 

Page 38, second to the last 
bullet: "Since the SBAC 
assessment intends to allow 
an item to assess multiple 
standards ..." "Item" needs 
to be defined in this context. 
I'm assuming that the 
reference here is to writing 
prompt, performance task, 
or extended response item, 
but not MC items. 

Test Design/State 3 

The use of "item" for the 
purposes of the study's 
guiding principles is used in 
its broadest terms. The work 
groups, particularly the item 
development work group, 
will need to discuss and 
define what is intended by 
"item" in this context. 




146. 

Why print mathematics in 
lower case when Algebra 
and Geometry are in upper 
case; the same goes for High 
School. 

State 4 

This is a style issue 




147. 

The second paragraph 
repeats eligible content for 
Summative Assessment (this 
should be explained in the 
first paragraph. 

State 4 

This has been addressed in 
the final report 
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148. 

It is clear that the purpose of 
the study was to analyze 
standards, determine DOK 
ranges within the standards, 
types of items that would be 
suitable for assessment of 
specific standards, criteria to 
determine eligibility and 
eligibility of items for 
assessment. However, was 
the intent of the report to 
inform us of the process? or 
to initiate discussion? 

State 5 

The intent of the report is to 
initiate/inform discussion. 
Clarification has been 
provided in the final report 




149. 

Why are oral response items 
(page 15) classified as 
measureable with on- 
demand but OR not listed as 
an item type? OR discussed 
in several places. Needs 
clarity. 

State 5 

This has been addressed in 
the final report 




150. 

Criteria were clear. 
Protocol— How many people 
were actually involved? 

Were those listed table 
leaders, reviewers, or all 
inclusive? Implications? 

State 5 

This has been addressed in 
the final report 




151. 

The charts are clear, but 
even with the Appendix D, 
references to the codes took 
some time. What are the 
intended uses of the DOK 
ranges— how will alignment 
be determined? 

State 5 

This has been addressed in 
the final report 




152. 

Pg 11- first line: delete "to" 

State 5 

This correction has been 
made 




153. 

Pg 26— first line of 2 nd 
paragraph: 

spacing/capitalization 

State 5 

This has been addressed in 
the final report 




154. 

It would have been nice to 
know the specific "eligible 
content" standards that will 
be included, and the specific 

State 6 

Response to this issue is 
beyond the scope of this 
project/report. 

Suggest follow up by 

X 
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U 

< 

H 


content that will be 
excluded from the 
development process. 


consortium's work 
groups/committees: test 
design; item development; 
performance tasks; 
administration; accessibility 
and accommodations; 
technology approach; 
formative processes and 
tools/professional 
development; content 
specifications 




155. 

given that multiple item 
types are being developed, 
it seems difficult to me that 
there are some standards 
that cannot be assessed 
using one of the proposed 
item types, although 1 
suppose there are some that 
would require too much 
time to do, or too much 
time to score, thereby 
making an item cost- 
restrictive. 

State 6 

Readers of this report 
should refer to the criteria 
used for determining how 
item type was assigned to 
each standards (in the 
report's Methodology 
section). 

Additionally, there should be 
follow up by consortium's 
work groups: test design; 
item development; 
performance tasks; 
administration; accessibility 
and accommodations; 
technology approach; 
formative processes and 
tools/professional 
development 

X 



156. 

for states that want to 
augment the tests with their 
own standards (1 think up to 
15% is allowed?), what is the 
mechanism for that to be 
done, and who would be 
responsible for submitting 
such items? 

State 6 

Response to this issue is 
beyond the scope of this 
project/report. 

Suggest follow up by 
consortium's work groups: 
test design; item 
development; performance 
tasks; administration; 
accessibility and 
accommodations; 
technology approach; 
formative processes and 
tools/professional 
development. Discussion 

X 

X 

X 
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with the consortium's 
Executive Committee and 
TAC is recommended 




157. 

On page 16, it might be 
good to include a 
description of the DOK levels 
below Table 9-like on page 
35 for the comment codes. 

State 7 

This has been addressed in 
the final report 




158. 

On p 7, need to define what 
you mean by "learnable" 
within a school 
year.. ..sufficient time for the 
average student to 
reasonably learn this 
content within a school year 
or course?" Make sure to 
add "course" in all such 
places as we consider 
courses and not school year 
as the unit of analysis in HS. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




159. 

Similarly, p 8, you might 
want to provide more 
information about what 
"expected" means .... Is this 
content essential? Core? 
Critical? Valued? 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




160. 

Recommended global 
change: 1 would watch 
references to "coded" or 
"codings" throughout 
report.... 1 would use "rated" 
or "ratings" instead so it is 
clear this was a judgment 
call. Eg, p 7, coded is used 
twice in section Study 
Criteria, also p 10 (codings 
and Coding Protocol). 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




161. 

Pg 8-9. Readers may not be 
clear on difference among 
item types... .need to refer 
them here to Appendix B. A 
couple of your decision rules 
provide some clarification, 
but readers will expect a link 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 
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to this information when the 
item types are first 
presented. 






162. 

Rationale for "standardized" 
comments is needed to 
ensure transparency.. ..we 
don't want the reader to 
think, for example, that we 
were trying to steer findings 
or otherwise influence 
raters. ..need to explain both 
in this section and on p. 33 
(Implications) 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




163. 

May want to make it clear 
from the outset that all 
eligibility criteria are 
weighted equally. Similarly, 
you do get close to saying 
that this is a conjunctive 
model (not compensatory), 
but you will want to 
reinforce this as, like AYP, it 
has implications for findings 
(i.e, it is all or nothing; one 
"no" and you are out) 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




164. 

Per comment above, need 
to reinforce that this is a 
conjunctive, not 
compensatory, model 
(maybe on p. 14, with 
Overall Findings). Then you 
need to stick to this rule in 
your findings. ..on p 14 
bottom you introduce the 
notion of 

measurable/eligible. ...don't 
they have to meet all 3 
criteria to be considered 
eligible? 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




165. 

P. 15 under Item Type, 
second line: add 
effectively. .."which could be 
most effectively and 
efficiently used 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 
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166. 

Top of pg 17, need table 
reference, le, "Overall, as 
shown in Table 10(?), the 
results for eligible item 
types...." 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




167. 

2 nd paragraph on p 17 goes 
beyond findings, belongs in 
implications 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




168. 

See note above about 
rationale for standardized 
comments, need to 
reinforce here 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




169. 

P35: need some sort of 
introduction under 
Examples of Use of Data. 
Also recommend changing 
to Examples of Effective 
and/or Appropriate Uses of 
Data. The value of this 
section is unclear to me as 
is. Are these scenarios? 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




170. 

One last step that might be 
really helpful is if to look 
more closely at the 
characteristics of the 
ineligible content in each 
subject area to see if you 
can draw any 
generalizations. SBAC 
leaders and developers will 
need to pay attention to 
such generalizations as they 
will need to find strategies 
for measuring that content 
with their other 
assessments. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




171. 

References to the focus on 
the summative assessment 
are consistent throughout 
report with the exception of 
first paragraph on p. 1 
("various 

assessments"). ...recommend 
changing that text to 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 
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"summative assessments at 
each grade" or something 
like that so focus remains 
clear. 






172. 

P.l Parenthetical statement 
under Purpose of Eligible 
Content Project needs to be 
removed or clarified, does 
not fit. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




173. 

In some cases 1 found the 
wording in your email 
slightly clearer than in the 
report itself— eg, 
grades/courses for math 
were clearer in email than 
on p.l under Purpose of 
Eligible Content Project. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




174. 

Headings sometimes too 
specific, not inclusive of all 
content that follows. For 
example, on p. 1, a better 
heading would be "Focus for 
this Study" rather than Key 
Questions. On p 5, a better 
heading might be "Purpose 
of SBAC Assessment 
System." Sure seem to be a 
lot of headings in this 
report, with just a few 
sentences under each. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




175. 

Key question 1 on p.l: need 
to specify what you mean by 
"eligible".... Clearer version 
might be "Using specific 
criteria for 'eligibility', which 
CCSS can be appropriately 
measured by the SBAC 
summative assessment at 
each grade?" Then refer 
reader to Methodology (p 7, 
9) or Appendix A (p 40) 
where you define your 
criteria. Eligibility is a 
judgment call that can vary, 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 
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Reviewer Comment 

Comment Source 

Action/Response from 
Report's Authors 

Recommended 

Follow-Up 

Work Group(s) 

Executive 

Committee 

TAC 


depending on the criteria 
used. 






176. 

Table 2 on p 2 does not add 
value, Table 1 is sufficient. 
That level of information 
may have been useful in an 
internal protocol for analysts 
but not needed in this 
report. Similarly, 1 would 
remove the domain codes in 
Tables 3, 4, & 5. ..these fit in 
an internal protocol, not 
final report. 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 




177. 

Value of naming and 
providing long description of 
analysts in body of report is 
questionable. You can 
include resumes as appendix 
if SBAC requested 
information about 
qualifications of analysts. 

PMP member involved in 
SBAC proposal and research 
activities 

Guidance provided to the 
authors of the report by 
those commissioning the 
report was to include a 
description of the analysts 
because of concern raised 
about their 

backgrounds/qualifications 




178. 

Global concern: be 
consistent throughout 
report. Use "ineligible" or 
"not eligible" consistently. 1 
prefer ineligible, (e.g., top of 
p. 17) 

PMP member involved in 
SBAC proposal and research 
activities 

This has been addressed in 
the final report 
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APPENDIX G. CONSIDERATIONS FOR NEXT STEPS— SUGGESTIONS BASED ON COMMENTS FROM THE 

CONSORTIUM'S TECHNICAL ADVISORY COMMITTEE 


The following suggestions are based on selected comments from members of the Consortium's 
Technical Advisory Committee. These suggestions are intended to generate and/or guide further 
discussion of this study's data among Consortium work groups and committees (e.g., Test Design, Item 
Development, and Accessibility and Accommodations Work Groups, content specifications committee). 

Note: Other comments from Technical Advisory Committee members that are not presented here were 
addressed, as possible and appropriate, in the text of the report. 

A Validity Framework for Test Design and Item Development that Ensures Appropriate Access to All 
Students 

As the Consortium moves forward with its summative test and item design and development, it is critical 
that it consider the following: 

1. Specify in greater detail what the students are to learn. 

2. Specify in greater detail the claims to be made about the students in terms of constructs and 
degrees of expertise, and then design the assessments to provide that type of evidence. 

Doing so will help to establish critical cognitive models for the assessed domains (i.e., ELA, mathematics) 
that will inform the subsequent development of an assessment framework and theory of action that can 
guide the development of the summative assessment and its items (B. Gong, personal communication, 
February 21, 2011; Mislevy, Almond, & Lukas, 2003; Mislevy & Haertel, 2006). 

While considering what students are to learn and the claims to be made about students from the results 
of the summative assessment, it is essential that the characteristics of the students be understood and 
considered— that is, the characteristics of all students who will be administered the summative 
assessment, including English learner (EL) students and students with disabilities (SWDs). These 
students' capacities as well as their challenges must be considered upfront and throughout the test and 
item design and development processes. EL students and SWDs have characteristics (e.g., sensory, 
cognitive, physical, linguistic, socio-cultural) that require certain supports (via accessibility strategies 
and/or accommodations) that interact with assessment content, item formats, and administration 
conditions, for example. Therefore, in order to best ensure the development of a summative 
assessment that yields valid interpretations and appropriate consequences, design and development 
discussions must always consider the students. 
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The Domain-Sampling Approach and Implications for Test Design and Item Development 

The figure below contrasts two fundamentally different ways of designing an assessment: domain- 
sampling and learning progression. 



Using the data provided by this study, Consortium work groups can consider the following three 
activities that relate to the domain-sampling approach for summative assessment design. Each activity is 
an increasingly sophisticated alternative to domain sampling. 

1. Organize eligible content by "domain clusters." 

Considering the characteristics (e.g., DOK) of related standards (e.g., comment codes 305 and 306), 
determine reasonable domain clusters (the shapes that contain the standards) and then, as a first step, 
ask "What sorts of items, and how many, would we need to create in order to elicit the kind of 
information that would help us decide if a student was proficient or had mastered the content in the 
domain cluster?" A domain cluster approach, rather than a standard-by-standard approach, to designing 
the assessment would lend greater coherence to the task of developing an item framework. 

2. Follow domain clusters over grades. 

An alternative to #1 is to follow domain clusters that appear in successive grades (e.g., see Table 27), 
Sample Set of College and Career Readiness Anchor Standards Across Grade Levels), and then consider 
item development in terms of the information that would need to be elicited to best determine 
"proficiency" in this domain in grade X versus grade X+l. Some educators may consider this a learning 
progression; however, it only lays out a scope and sequence of clustered standards without explicating 
any kind of underlying cognitive model. Nonetheless, this approach represents an improvement over #1 
in that items are being developed with some general design or theory for how mastery is changing over 
time. 
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3. Merge related domain clusters across and over grades; combine with process skills. 


This alternative is the most demanding approach of the three. Fundamental to this approach are that (1) 
the CCSS are viewed as a starting point for item development; (2) there are many different ways that 
items can be conceptualized such that they measure fewer big things (e.g., knowledge, skills, abilities); 
and (3) items should lend themselves to greater accuracy of measurement. Applying this approach to 
test and item design, for example, domain clusters that are considered separately in an earlier grade 
could be better merged at later grades. 
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